: How to add favicon to robots.txt How can I disallow this URL: http://www.example.com/assets/ico/favicon.png? Would these lines in my robots.txt work? User-agent: * Disallow: /blah/assets* Disallow: /blah/assets/ico/

@Cofer257

Firstly, you shouldn't block your favicon image unless you have a very good reason.

What exactly is the 'blah' supposed to represent in your example? If the URL is as you state in the question, you should just use:

Disallow: /assets/ico/favicon.png


That will prevent crawling of the specific image. To block the whole ico directory, use:

Disallow: /assets/ico/


In robots.txt, any path listed matches URLs starting only from the beginning, so the above matches anything beginning with /assets/ico/.

There is no such thing as a * wildcard for Disallow in the robots.txt standard. It is a non-standard extension that according to Wikipedia is only supported by Google.

Vote Up Vote Down

@Odierno851

Both of those would work, but they would also block anything else in those directories. If you only want to block that specific URL, you could do:

Disallow: */blah/assets/ico/favicon.png


Do you have a Google Webmasters Account setup? If so they have a new robots.txt test page where you can see if a URL is blocked or not based on your robots file.

EDIT: My original answer had the below. This would block all URLs that end with favicon.png since it has the $ at the end, but it's pointless in this case since only one favicon.png file can be in that directory. support.google.com/webmasters/answer/6062596?hl=en&ref_topic=6061961
Disallow: */blah/assets/ico/favicon.png$

Vote Up Vote Down