Search…

X3 Photo Gallery Support Forums

Search…
 
kmorwath
Topic Author
Posts: 8
Joined: 23 Jan 2020, 15:11

Upload of more photos than a couple fails with 500 error

24 Jan 2020, 13:54

Hi, I'm testing X3 to see if it fits my needs. So far, everything is fine but uploads. When I try to upload more than a couple of photos (15-20), the download starts, but after a while it looks stuck for a while (upload data aren't updated), then one image after another shows a "server error" or "unknown error" (PHP set to log to a file, but no errors are written).

It looks that X3 tries to process all the image in parallel. I have a slow upload connection (about 500-600 kbps, which I cannot improve because the broadband network in my town has been delayed until 2021...), and while my test server (a cloud hosted VPS) has good connectivity, it does not have much computing resources, but I don't have issues running Drupal 8 and RoundCube from it. Server is running Windows 2019 (IIS 10), PHP 7.3.13 (64 bit), X3.27.6. X3 is set to run in its own application pool.

Is X3 really trying to upload all the images a once? Is there a way to reduce the number of images processed at once? I tried to set to 2 the concurrent uploads in PHP, and to increase to 90s the max script run time, without resolving the issue. The max memory setting is 192M, should it be increased?

Thank you.
 
User avatar
mjau-mjau
X3 Wizard
Posts: 13993
Joined: 30 Sep 2006, 03:37

Re: Upload of more photos than a couple fails with 500 error

25 Jan 2020, 02:59

What are your PHP settings for max_file_uploads, post_max_size and upload_max_filesize? You can find these values by going to /?diagnostics in your X3 page, or by going to panel > Tools > PHPinfo. The error you are having, is most certainly related to your settings for the above and the amount of images you are attempting to upload in a single chunk.

It's got nothing to do with your own internet connection, and it's not related to server resources (X3 runs fine on most low-resource shared hosting also). But it is related to your PHP settings, or in some obscure case, some other error on your server (but I doubt it).

I can only diagnose further if you can offer a login to your panel so I can check, or at least let me know the PHP info fro the above.
 
kmorwath
Topic Author
Posts: 8
Joined: 23 Jan 2020, 15:11

Re: Upload of more photos than a couple fails with 500 error

25 Jan 2020, 13:01

Hi, thank you for you answer. Here the data:
  • max_file_uploads: 2 (it was 20 when I got the error fist, I tried to reduce it)
  • post_max_size: 8M (I'm uploading JPEG images no larger than 6M), I'll try to increase it anyway.
  • upload_max_filesize: 16M
While I'm still trying to get PHP errors if any, I found many of these kind of lines in the httperror.log file:

HTTP/2 POST /photo/panel/upload.php 79 - 2 Timer_EntityBody photo
HTTP/2 POST /photo/panel/upload.php 49 - 2 Timer_ConnectionIdle photo

Don't know if they could help. I'm trying to upload about 15-20 images with sizes from about 1M to 6M. Having to upload just a couple each time makes the upload process a bit cumbersome, especially now I exported albums from Canon Irista and trying to re-created them in X3. Is there a way to upload via FTP and then import them into a "gallery"?

Being a test setup, I can give you access to the panel to test it, maybe my setup was wrong, I can set a temporary password and let you access it, just I would prefer not to post it on a publicly available site.
 
kmorwath
Topic Author
Posts: 8
Joined: 23 Jan 2020, 15:11

Re: Upload of more photos than a couple fails with 500 error

25 Jan 2020, 13:19

I tried to increase both post_max_size and upload_max_filesize to 64M but it didn't help. I can upload up to 3 photos at once, but with 4 I start to get errors.
I forgot to add that I tried with the resize option both enabled and disabled. I also put all the directory used by the site and the upload in AV exclusion, with little or no changes.
 
User avatar
mjau-mjau
X3 Wizard
Posts: 13993
Joined: 30 Sep 2006, 03:37

Re: Upload of more photos than a couple fails with 500 error

25 Jan 2020, 22:06

kmorwath wrote:max_file_uploads: 2 (it was 20 when I got the error fist, I tried to reduce it)
I don't quite understand why you have reduced it, as it is the most important setting, likely related to your issue. This number needs to be high to support simultaneous upload of multiple images via HTTP on your server. Often, the limit is set to 20 (by default and/or on shared servers) ... We have our set to 100 on our flamepix.com service, to accommodate extensive image-uploading. The number you have here will limit your uploads-amount ... That is after all the entire point of this setting!
kmorwath wrote:[*]post_max_size: 8M (I'm uploading JPEG images no larger than 6M), I'll try to increase it anyway.
[*]upload_max_filesize: 16M
Probably unrelated, although both these settings should be at least as large as the largest single image you intend to upload.
kmorwath wrote:While I'm still trying to get PHP errors if any, I found many of these kind of lines in the httperror.log file:
HTTP/2 POST /photo/panel/upload.php 79 - 2 Timer_EntityBody photo
HTTP/2 POST /photo/panel/upload.php 49 - 2 Timer_ConnectionIdle photo
It's just an "open connection", made on upload, which is not closed (yet). Doesn't really say much, and it's nothing directly related.
https://serverfault.com/questions/20209 ... ectionidle
kmorwath wrote:Don't know if they could help. I'm trying to upload about 15-20 images with sizes from about 1M to 6M. Having to upload just a couple each time makes the upload process a bit cumbersome, especially now I exported albums from Canon Irista and trying to re-created them in X3.
Sure. But first of all, please try to set your max_file_uploads to a HIGH number (for example 100).
kmorwath wrote:Is there a way to upload via FTP and then import them into a "gallery"?
Yes, you can upload images directly into galleries via FTP. After upload, you would need to view the folder(s) from the X3 panel, and click "save" at least once, to make sure page-cache is refreshed.

Also, when uploading via FTP, you should make sure your images are resized appropriately for the web prior to upload. For example 1600 px or 2000 px ...
kmorwath wrote:Being a test setup, I can give you access to the panel to test it, maybe my setup was wrong, I can set a temporary password and let you access it, just I would prefer not to post it on a publicly available site.
I can diagnose further if necessary, but let's see what you make of the above first.
 
kmorwath
Topic Author
Posts: 8
Joined: 23 Jan 2020, 15:11

Re: Upload of more photos than a couple fails with 500 error

26 Jan 2020, 12:57

Hi, thank you for the time you're spending on this issue. I reduced the max_file_uploads value hoping it would reduce the number of parallel uploads, and kept the other files in queue, but obviously it doesn't work that way. I set it to 100, but nothing changed, I still get errors.
Then I tried to increase to 600s max_execution_time and max_input_time, to see if it could be a processing timeout error, but again got errors. Then I also increased the shutdown time limit of the application pool, but it didn't help either.
I'll try a FTP upload, although being able to upload via HTTP is especially useful when I'm traveling. Resizing is not an issue, I can make a Lightroom export preset and use it anytime I need to prepare images to be uploaded.
 
User avatar
mjau-mjau
X3 Wizard
Posts: 13993
Joined: 30 Sep 2006, 03:37

Re: Upload of more photos than a couple fails with 500 error

26 Jan 2020, 20:11

I can only diagnose further if you can offer login to your panel so I can test from here.
 
kmorwath
Topic Author
Posts: 8
Joined: 23 Jan 2020, 15:11

Re: Upload of more photos than a couple fails with 500 error

27 Jan 2020, 14:01

OK, I bought a private license because I didn't want to abuse your help without buying the product. I believed it was a simple issue fixable with some settings tuning, but it looks it is not.

Anyway I tried the FTP upload method and it works simply enough to be fully usable to upload many photos - given my slow upload speed I prefer to upload them in a batch when I'm doing something else. And I like the content display so it's fully usable for my needs even if the web upload has issues.

Just hope you won't release a new version in the upcoming weeks - hope you have a special upgrade prices in such situations :)

Is it OK if I use the contact form on the photo.gallery site to send you login data?
 
User avatar
mjau-mjau
X3 Wizard
Posts: 13993
Joined: 30 Sep 2006, 03:37

Re: Upload of more photos than a couple fails with 500 error

27 Jan 2020, 20:42

Thanks!
kmorwath wrote:I believed it was a simple issue fixable with some settings tuning, but it looks it is not.
If you can't even upload at your max_file_uploads limit, then it sounds like a server/network related issue. I would need to test.
kmorwath wrote:Just hope you won't release a new version in the upcoming weeks - hope you have a special upgrade prices in such situations :)
All upgrades are FREE.
kmorwath wrote:Is it OK if I use the contact form on the photo.gallery site to send you login data?
Yes. Or send private message from this forum.
 
User avatar
IFireflyl
Posts: 6
Joined: 15 Mar 2020, 05:56

Re: Upload of more photos than a couple fails with 500 error

23 Mar 2020, 09:46

Has there been any update on this? My uploads occassionally fail as well. The only thing I found was that if I look at the console in "Inspect Element" (I'm using Firefox) I see this error:

Source map error: Error: NetworkError when attempting to fetch resource.
Resource URL: moz-extension://f9038f33-16bf-4bda-8894-a64194059ffd/inline/injected.js
Source Map URL: injected.js.map
Image
 
User avatar
mjau-mjau
X3 Wizard
Posts: 13993
Joined: 30 Sep 2006, 03:37

Re: Upload of more photos than a couple fails with 500 error

24 Mar 2020, 00:50

IFireflyl wrote: Has there been any update on this? My uploads occassionally fail as well. The only thing I found was that if I look at the console in "Inspect Element" (I'm using Firefox) I see this error:
Sorry, not really. This is an issue that can stem from many things on server or network, including PHP max_file_uploads Firewall, Network connectivity, missing server/PHP extensions (for example encoding of special characters). I have never heard of the issue in the original page "it looks like X3 tries to upload all images in parallel" ... The X3 uploader will upload at max capacity, negotiated between the client and the server. The server should manage the throttling of uploading. Besides, it's not like uploading is such an intensive task ...

You have a link/login? I could do some basic diagnostics at least. I would like to see all PHP settings, and check if I can reproduce the issue from here.
IFireflyl wrote:Source map error: Error: NetworkError when attempting to fetch resource.
Resource URL: moz-extension://f9038f33-16bf-4bda-8894-a64194059ffd/inline/injected.js
Source Map URL: injected.js.map
The above is some kinda unrelated browser plugin. You won't find errors in the javascript console that explain your upload issue anyway, because it happens on server.
 
User avatar
mjau-mjau
X3 Wizard
Posts: 13993
Joined: 30 Sep 2006, 03:37

Re: Upload of more photos than a couple fails with 500 error

24 Mar 2020, 00:51

Edit: I imagine it could be something related to this:
https://forum.joomla.org/viewtopic.php?t=870082

You should check your apache error logs.