Photo Gallery Forums

I installed the latest version of X3 on www.merelyafleshwound.com yesterday.

When I crawl my site using the ScreamingFrog SEO Spider I am getting 5,586 404 Page Not Found errors. I think there is one for every image on the site.

Here is one example...

Type: AHREF
From: http://www.merelyafleshwound.com/mounta ... /columbia/
To: http://www.merelyafleshwound.com/mounta ... mboldt-13/

Any ideas why this is happening?

alexhenes wrote:Any ideas why this is happening?

Yes. It seems there was an issue because all your images end with *.JPG in uppercase, and these files were not detected properly. The issue is fixed already, and you can just update the following file:
/app/stacey.inc.php

alexhenes wrote:When I crawl my site using the ScreamingFrog SEO Spider I am getting 5,586 404 Page Not Found errors. I think there is one for every image on the site.

On an unrelated note, I'm impressed that your server doesn't buckle under from this process. Some X3 users have problems just getting a single new page to re-process. In your case, you are sending a spider to process 5586 pages as fast as possible ... All these pages require processing (menu, content, page-creation-to-cache). How many seconds does it take to create a single un-cached page for you after you change something in the panel? Multiply that result with 5586, and you have the task your server is asked to process. Keep in mind, the image-pages aren't normally visited by humans, unless they "open link in new tab", and is mostly reserved for landing-pages for sharing and SEO. Therefore, the pages will likely get processed progressively over time in real-life.

To be honest, the task might be much more heavy after the fix. Why? Because in your current case, X3 is just returning a cached 404 page for each result. In the case it works, it will need to process the actual page.

Just a heads-up, so you know what is going on ...

Thanks... all my 404s are gone!!!

When I initially started crawling the site I did crash the server... ha! Now I crawl with just 1 thread.... don't want to piss off my server mate.

I have crawled with up to 3 threads and things seem to work fine.... more than that is dicey.

ScreamingFrog is a great tool to see site issues, response times, etc.

I also use it to create my cache file for various image sizes. I crawled the site with various user agents... ie. iPhone, Android Phone, iPad, Android Tablet, Googlebot Desktop agent, etc. A great way to quickly build your cache.

Site performance has significantly improved since the X3 beta first started. Here are the stats for the crawl I just did with the bug fix.

Pages Crawled = 12,498
Average Response Time = 0.39 secs
99.77% of pages were sub second

Here are some more details

After building cache...

Avg Response Time = 0.23

Nice. We did some improvements in the code, and I'm happy to see it's faster. I think primarily it's related to the "menu fragment cache" we now use. Instead of each page having to create/process the menu from scratch, it gets created and cached on the first page visited (after change), and the other pages can use the cached menu fragment. The menu-creation process is the heaviest process, so it should help a lot.

Honestly, it seems your server is very fast considering your relatively heavy structure. We have plans for further performance improvements in the code, because there are still some slow processes that run when validating the cache.

alexhenes wrote: I also use it to create my cache file for various image sizes. I crawled the site with various user agents... ie. iPhone, Android Phone, iPad, Android Tablet, Googlebot Desktop agent, etc. A great way to quickly build your cache.

Just a comment about this ... Keep in mind, there is no difference between the cached pages across different devices. The page output is the same, and instead it's the "responsive" frontend (javascript+css) that modifies layout based on device. However, different devices will likely request different image-sizes, but I doubt the crawl bots load images in the first place ... The bots would have to have javascript enabled, which would be highly unusual. Therefore, I would conclude it unnecessary to crawl the website with multiple agents.

Cool... thanks Karl.

The only performance related issue I am seeing is that it takes about 30 seconds for the three bars in the mobile menu to load. In addition the first image to appear in mobile also takes about 30 seconds. When the image does appear it is about 3-4 images into the slideshow. This happens if I am on wifi or on 4G

Alex, why do you have multiple insanely massive images in your start page slideshow?


Each image is around 8MB. That's a whopping 30MB to load just for the first images in the slideshow for the start page, so no wonder it loads slowly on mobile. This will obviously steal all network resources, partially blocking other scripts (for example icons in menu) from loading effectively also. It's slow on desktop also because of this, but you may not have noticed if some resources were cached.

You need to use the upload-resizer, or resize images prior to upload.

will do...

What is the max size you recommend?

Btw, you have disabled the CDN option? May I ask for what reason? No wonder the icons load even slower, since you are trying to load everything from your own local server in addition to the heavy images. If you used the CDN, many resources (scripts, css, icons) would be loaded in PARALLEL with images, instead of being queued from the same server.

alexhenes wrote:What is the max size you recommend?

Without going into detail: 1600px, as we have set as default in the uploader resizer. I was hoping that those who turn OFF the upload resizer, do it because they have already resized their images, and/or are technically aware of the implications involved.

From our demo gallery, with images around 2-400kb, which is 4% the size of your images:


Resizing images is very important, and Im surprised you were not aware that all these massive images would basically block the network. Imagine the visitors who try to visit the site on 3G, or from mediocre internet speeds.

All fixed! The only images that were that large were the main slideshow. All others were re-sized. I also enabled CDN and pre-load site. Mobile performance is much better! Thanks

I just crawled another domain I have X3 installed on... www.39thavenueapartments.com and am getting 7 404 errors. One example is...

From: http://www.39thavenueapartments.com/
To: http://www.39thavenueapartments.com/39th-02/

I installed the same fix to this domain as you recommended for www.merelyafleshwound.com

alexhenes wrote:From: http://www.39thavenueapartments.com/
To: http://www.39thavenueapartments.com/39th-02/

Where can I find that page in the menu? I need to see that this is a valid page ... if not then obviously it will return 404 page not found.

I don't think it is a page... it is an image in the slideshow minus the .JPG

alexhenes wrote:I don't think it is a page... it is an image in the slideshow minus the .JPG

Are you using the updated file /app/stacey.inc.php as noted earlier in this thread? I am not 100% sure what version we are dealing with here to be honest. Can you try to rename the file to *.jpg (lowercase)? This will allow us to know exactly what the issue is.

images renamed to *.jpg (lower case). Still seeing the same 404 errors in the crawl. All errors stem from the www.39thavenueapartments.com and point to a directory with the name of the images in the slideshow minus to .jpg