: Google Webmaster Tools robots test not working Within Webmaster Tools I have supplied my test content: User-agent: * Disallow:/admin/ Disallow: /tag/ When I specify the URL to test against, for

@Dunderdale272

If you've just updated your robots.txt file then Google may not have re-read it yet. Give it some time to get re-crawled. Do make sure you have a newline between User-agent: * and Disallow:/admin/.

You could also ask Google to permanently remove specific directories using the URL Removal Tool. From Google's help page:


On the Webmaster Tools home page, click the site you want.
On the left-hand menu, click Optimization and then click Remove URLs.
Click New removal request.
Type the URL of the directory you want removed from search results
and then click Continue. (The URL is case-sensitive)
Click Remove directory.
Click Submit Request.


Also take a look at when not to use the URL Removal Tool.

Vote Up Vote Down

@Yeniel560

User-agent: *
Disallow: /admin/
Disallow: /tag/


Try this, it must work.

Vote Up Vote Down