X3Photo Gallery Forums

Hello people! Because there has long been talks about images not being able to be indexed by Google, I finally took some time to research the option of including a sitemap.xml with reference to all images. While researching, I came across a few challenges, which raise some questions.

Just to be clear, when we speak of "indexing images in search engines", we are speaking of images that may appear in Google search after clicking the "images" tab from the Google search page [screenshot]. The normal Google search will display pages/info only [screenshot], although sometimes it may display a preview section of images if it thinks the user is specifically searching for images [screenshot].

The problem.
As noted across various previous forum topics, full-size images in X3 might not get indexed by Google. This would be caused by the image opening in POPUP when clicked (useful for humans of course), which Google does not consider, and therefore only recognizes the containing page (which may contain many images of course). I did some tests, and I found that Google does in fact find images, and can identify them from search, but the link to the image is the parent container page, which is not unique for the searched image of course. Example search "site:demo.photo.gallery nature cone" [screenshot], will find the cone photo, although the image may be a downsized version, and it will belong to the gallery that contains the image. Google will ignore the "landing page" for the image [screenshot], because it simply doesn't arrive on that page by click.

1. Google can't necessarily find the full-size image (because it's opened by Javascript)
2. Google doesn't know that there is an "image landing page" dedicated to the image, because the popup will override the link to the image landing page (which is way preferable from a human perspective).

Solution
I have already started work on a create_image_sitemap.php script, that creates a sitemap XML file with all images in an X3 website. At first, I added all images to be listed under the <url> of the gallery(page) that contains them. However, this seems like an incomplete solution ... Although it allows full-size images to be found by Google, it's would still be linking them to the gallery page, which has title/text related to the gallery, not specifically for the image. Also, the full-size image does not really exist on that page, until the user clicks an image (loads the image into popup). The most desired solution here, would probably be to link each image under it's own X3 "landing page" ... 

/content/galleries/nature/cones.jpg <image>
/galleries/nature/cones/ <loc>

This was of course one of the points of the X3 "landing page" in the first place, so that each image had it's own page, that contains meta data, title and text specifically for each image. Although I am not personally a fan of "landing pages" from a human-perspective, this is surely the best option from a SEO-perspective. As stated by Google, " ... the content and metadata of the pages where an image is embedded can have a great influence on how and where the image may appear in Google's search results.".

X3 image sitemap generator
Although I was first thinking of adding this to next X3 release (coming soon), I think it's best to keep it as a separate script for now. First of all, it can be a slow process to create a full sitemap from all images in all pages, and it's not useful to have this auto-created every time you make a small change in X3. Also, most users have no use for this function. The concept is that it would work like this:

• Trigger /panel/create_image_sitemap.php in browser.
• The script will only execute if you are logged in to panel.
• It will create a file /sitemap.xml in root of your X3 website, which would override the automatically created sitemap (which isn't actually there as a file).
• The sitemap will create an <image:image> for every single image in your X3 website, with each image linked inside an <url> to the image landing page (see code example above).

Additionally, there are some challenges with what images to include, but it would have to work like this:

• Images from "hidden pages" will still get listed in the sitemap, because these pages are still public and might be linked from other locations.
• Images from pages that are not in the menu will still get listed, because these pages are still public, and might get listed from folder overview pages, or linked to manually.
• Images from pages with Settings > Gallery > Hide Gallery enabled, will not get listed, because this option explicitly means you don't want to display the images on the page.
• Images that are specifically hidden, will not get listed.
• If a gallery shows images from another folder (assets), images will only get listed if the assets folder itself is public (in most cases, it is). For example, images in /examples/gallery/grid/ will have landing pages like /examples/assets/bubbles/.
• Images in folder names that start with "_" (underscore) will not get listed, because these pages are specifically hidden and will not load in browser.
• Images that start with double __underscore will be hidden, because this is how X3 works.
• Images in password-protected pages, I am not 100% sure, but I assume they should not get listed. This one is a bit complicated, because logins are recursive and request-based, and aren't specifically attached to each gallery/folder.

This script will be available as standalone at first, but I may consider implementing it into the X3 control panel. The script could possibly auto-submit sitemap.xml to Google after creation, or just list the sitemap in robots.txt.

Impact of additional pages indexed by Google
Adding all image landing pages to your sitemap/seo will of course mean that the amount of pages you want Google to index will increase drastically. There is an option Panel > Settings > Advanced > "Prevent search engines from indexing image pages", created specifically for some that are in the opinion that all the "image landing" pages are "washing out" the website's SEO effect. This would be a contradiction when adding all images to the sitemap. I haven't personally come across any evidence that indexing "more pages" is harmful in any way.

Some technical notes about sitemap.xml

• Maximum 50.000 url's [ref], where each image will be 2x urls (including the image).
• Google ignores <priority> and <changefreq> values [ref], so these will not be included in the script.
• Google uses the <lastmod> value only if it's consistently and verifiably (for example by comparing to the last modification of the page) accurate [ref]
• It was previously suggested to include tags <image:caption>, <image:geo_location>, <image:title> and <image:license>, but these tags are now deprecated (not used by Google) [ref].

Image sitemaps was discussed in these forum topics:

• XML Sitemap to include photo landing pages - possible?
• I got: working keywords, search and smart search
• Individual images SEO is terrible
• SEO-problem: nothing is found
• X3 Sitemap php
• Search robot indexation photo

Google sitemap XML references

• https://developers.google.com/search/do ... ld-sitemap
• https://developers.google.com/search/do ... e-sitemaps
• https://developers.google.com/search/do ... gle-images
• https://www.sitemaps.org/protocol.html

---

FEEDBACK WANTED! Thank you.
:clap: :star: :clap: :star:

Hello @ef_f @marco963. This is a feature I am currently working on. It will be available shortly, but this post includes a proposed plan of how the sitemap generator will work, and I am asking for feedback before I complete the task.

ed_f wrote:could you pleeease give an exact description of how to implement your "sitemap", which is useable also for not so bright ones in programming, html etc.? thanks a lot!

The script will be available soon, with instructions, and it won't be difficult to operate.

This post is if you have feedback about how the sitemap will be built based on my post above. It's not a "magical" script, but the idea would be that it should allow Google to easier index all your images and the image landing pages.

marco963 wrote: When I run https://mydomain/panel/create_image_sitemap.php and I get a “file not found” message.

As noted in my post, "I have already started work on a create_image_sitemap.php script". It will be available soon. This is a post to allow suggestions on how to best implement the sitemap.

metallissimus wrote:This is a huge privacy concern.

True that. The reason I responded "not 100% sure", is because I was thinking neither Google or any human (without login access) would be able to view the linked page anyway. But I guess there is still a chance that images might get indexed, even if their page url is inaccessible. I will definitely ignore password-protected pages.

I have released the official X3 Images Sitemap Creator script:
viewtopic.php?f=51&t=10651

I also backtracked on a few statements:

mjau-mjau wrote:Images from "hidden pages" will still get listed in the sitemap, because these pages are still public and might be linked from other locations.

Images in hidden pages will definitely NOT get listed in the sitemap.

mjau-mjau wrote:If a gallery shows images from another folder (assets), images will only get listed if the assets folder itself is public (in most cases, it is).

Images in assets folders will definitely get listed in the sitemap, as long as they are used in at least one public gallery.

mjau-mjau wrote:Images in password-protected pages, I am not 100% sure, but I assume they should not get listed. This one is a bit complicated, because logins are recursive and request-based, and aren't specifically attached to each gallery/folder.

Images in password-protected pages will definitely NOT get listed in the sitemap.

---
This post is now locked. Please use the new official announcement for comments.