: Nofollow doesn't stop crawling? In my website, most of the pages has a linkedin sharing button written in this way: <li> <a id="share_linkedin" href="/link/share/linkedin/Dance+Combo+%28Jazz%2C+Tap%2C+C

@Shelley277

According to Google:


In general, we don't follow them. This means that Google does not
transfer PageRank or anchor text across these links. Essentially,
using nofollow causes us to drop the target links from our overall
graph of the web. However, the target pages may still appear in our
index if other sites link to them without using nofollow, or if the
URLs are submitted to Google in a Sitemap.


Google doesn't gaurentee that they won't crawl the URL, just that the contents won't affect your PageRank. In fact, Wikipedia entry nofollow explicitly claims that Google does crawl "nofollow" links, but again, doesn't use the contents to affect your PageRank any.

The Restricted by robots.txt message is fine--it won't affect your PageRank either. It is informational only.

You could drop the "external" part of the attribute though; using the standard rel="nofollow" is probably safer for crawler parsers that aren't as robust.

Vote Up Vote Down

@Si4351233

"Restricted by robots.txt" errors in Google Webmaster.


In other words, your nofollow directive is being noted by Google, obeyed and thus not an error. Working as designed by Google and not anything to worry about.

Vote Up Vote Down

@Pope3001725

It's hard to say with 100% certainty but you probably have nothing wrong with your code. A link with nofollow will not be followed but if another page links to it, even from another website, Google will crawl it unless you block it through another means (robots.txt, x-robots-tag, etc). So Google obviously knows that pages exists and may want to crawl but is correctly not doing so because of your robots.txt and thus is reporting it to you). I wouldn't worry about since you're ultimately getting the results you want. Of course you should check to make sure all of your links are correctly nofollowed.

Vote Up Vote Down