: Does google find and understand my sitemap? I have a site where I use robots.txt to reference my sitemap, since it's not located under the default /sitemap.xml. However if I go to google webmaster

@Ann8826881

You'll need to submit the XML sitemap in Google Search Console (formerly Google Webmaster Tools) in order to verify this.

It's quite possible that Google is already picking up your sitemap, having seen the reference in robots.txt. (Since Google does support the Sitemap directive in robots.txt) You should be able to check your access logs to determine whether Googlebot has requested it. However, that doesn't tell you whether Google found any problems with it.

I tried this some time ago myself. With the XML sitemap referenced only in robots.txt, even with the sitemap in its "default" location ie. example.com/sitemap.xml, there is no acknowledgement from Google unless you explicitly add it to GSC.

The Sitemap: directive in robots.txt is supported by all major search engines, not just Google.

It's not necessary to submit the sitemap in GSC for Google to pick it up, but it is necessary if you want to report on and diagnose it.


The reason for this is, that I use robots.txt to reference it, since the the sitemap is hosted on S3.


However, whilst the Sitemap URL can be on a different host to the robots.txt file, if the URLs in the sitemap are referencing a different URL-path to where the sitemap is located then this could be a problem. From Sitemaps.org:


...all URLs listed in the Sitemap must use the same protocol ... and reside on the same host as the Sitemap. For instance, if the Sitemap is located at www.example.com/sitemap.xml, it can't include URLs from subdomain.example.com.

You might be able to get around this with Google by registering all properties in GSC, but can you verify S3? And this won't work with other search engines.

Vote Up Vote Down