: 'Sitemap contains urls which are blocked by robots.txt.' Warning - However robots.txt file doesn't appear to be blocking anything Our site is built on WordPress. Whilst in the development stage,

Posted in: #GoogleSearchConsole #RobotsTxt #Seo #Sitemap #Wordpress

Our site is built on WordPress.

Whilst in the development stage, we had the option to Discourage search engines from indexing this site checked (Settings Reading).

We have now made the site live and unchecked this option.

Yesterday I submitted a sitemap into Googles Search Console but a lot of the sitemap paths are coming up with the warning:

Sitemap contains URLs which are blocked by robots.txt.

As far as I can tell, none of the site (Apart from the /wp-admin/) URLs are being blocked by our robots.txt file. I have tested it in the Search Console and that is saying it is fine.

I read some articles which say that the file can be cached for a little while, but it has now been a day since submitting it.

Is there anything I am missing or can do to stop this warning being thrown?

Sitemap - www.justaccounts.com/sitemap_index.xml
Robots File:

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

A few examples of the URLs which are being shown as blocked:

www.example.com/page-sitemap.xml https://www.example.com/category-sitemap.xml www.example.com/attachment-sitemap.xml

10.01% popularity Vote Up Vote Down

: How to export sites from Google Search Console? How do I export sites from Google Webmaster Tools / Search Console so I can import them into Bing Webmaster Tools? Help appreciated.

@Rambettina238

Posted in: #GoogleSearchConsole

1 Comments

: What is more important: time to first byte, or overall load time? In terms of being rewarded with higher rankings on Google, what is a more important attribute for a website to have? A fast

@Rambettina238

Posted in: #PageSpeed

2 Comments

: Geo Redirection. How to redirect Aus IP's from US site to AUS site my business partner is running an e-commerce store since 3 years in the US. I'm starting the same in Australia ( both selling

@Rambettina238

Posted in: #Geotagging

0 Comments

: Cloud Hosting - Choice of host I am amateur when it comes to the choice of the web server. I need to host 3 low-to-medium traffic Wordpress websites and I'm looking at budget cloud server

@Rambettina238

Posted in: #CloudHosting #LookingForHosting #Virtualhost #WebHosting #Wordpress

0 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Radia820

In my case Google was refusing to use (without saying) the new robots.txt because of the Windows line endings. I used dos2unix command and submitted again.

10% popularity Vote Up Vote Down

Feed

: 'Sitemap contains urls which are blocked by robots.txt.' Warning - However robots.txt file doesn't appear to be blocking anything Our site is built on WordPress. Whilst in the development stage,

More posts by @Rambettina238

: How to export sites from Google Search Console? How do I export sites from Google Webmaster Tools / Search Console so I can import them into Bing Webmaster Tools? Help appreciated.

: What is more important: time to first byte, or overall load time? In terms of being rewarded with higher rankings on Google, what is a more important attribute for a website to have? A fast

: Geo Redirection. How to redirect Aus IP's from US site to AUS site my business partner is running an e-commerce store since 3 years in the US. I'm starting the same in Australia ( both selling

: Cloud Hosting - Choice of host I am amateur when it comes to the choice of the web server. I need to host 3 low-to-medium traffic Wordpress websites and I'm looking at budget cloud server

Login to post a comment!

1 Comments

Back to top | Use Dark Theme