: Include "www" in sitemap URL in robots.txt when site redirects to no-www? Website is redirected to remove the www so www.example.com is redirecting to example.com. Question is do I need to include

@Harper822

In your situation, Don't include "www" because web crawlers that do read the sitemap directive from your robots.txt will attempt to access the "www" version of your sitemap and then be redirected to the non-www version of the URL.

Not only will you be wasting a couple extra bytes by including "www", but the crawlers that parse the sitemap directive properly will be making two requests to your server: one for the www version, and one for the non-www version.

Vote Up Vote Down