sitemapindex.xml

robots.txt containts following instructions:

User-agent: *
Allow: /public/
Disallow: /
Sitemap: sitedomain.com/sitemapindex.xml

it seems that in this case sitemapindex won't be accessible for the search crawler and should be moved to public folder, is it correct or crawler will process the sitemap in any case?

10.01% popularity Vote Up Vote Down

: SEO for company blog - subdomain + SSL or subdirectory without SSL I got an SEO question related to an issue we are having with how we set up our WordPress blog. It revolves around the benefits/penalties

@Cofer257

Posted in: #Blog #Https #Seo

1 Comments

: What is the purpose of a hidden rel=dofollow link? I recently took over webmaster duties for a website and found the following item repeated on every page of the site. <div style="position:

@Cofer257

Posted in: #Seo

1 Comments

: Using MongoDB objectID in query parameter, does it affect SEO? If we use object id of MongoDB in query parameter like for example, In browser URL, http://example.com/get-details/507f191e810c19729de860ea

@Cofer257

Posted in: #Mongodb #Seo

1 Comments

: GoogleBot supports localStorage, according to this service: https://www.whatismybrowser.com/ I found it, ironically, in the google-crawled version of the site. https://twitter.com/dcorbacho/status/702150914023538688

@Cofer257

0 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Jamie184

sitemapindex won't be accessible for the search crawler and should be moved to public folder

Yes, that is correct, the XML sitemap won't be crawlable unless moved to the /public/ folder.

Alternatively, you could add an exception for your sitemap:

Allow: /sitemapindex.xml

10% popularity Vote Up Vote Down

Feed

: Robots.txt disallow all except one directory and access to sitemap robots.txt containts following instructions: User-agent: * Allow: /public/ Disallow: / Sitemap: http://sitedomain.com/sitemapindex.xml

More posts by @Cofer257

: SEO for company blog - subdomain + SSL or subdirectory without SSL I got an SEO question related to an issue we are having with how we set up our WordPress blog. It revolves around the benefits/penalties

: What is the purpose of a hidden rel=dofollow link? I recently took over webmaster duties for a website and found the following item repeated on every page of the site. <div style="position:

: Using MongoDB objectID in query parameter, does it affect SEO? If we use object id of MongoDB in query parameter like for example, In browser URL, http://example.com/get-details/507f191e810c19729de860ea

: GoogleBot supports localStorage, according to this service: https://www.whatismybrowser.com/ I found it, ironically, in the google-crawled version of the site. https://twitter.com/dcorbacho/status/702150914023538688

Login to post a comment!

1 Comments

Back to top | Use Dark Theme