Mobile app version of vmapp.org
Login or Join
Connie744

: Does a link rel canonical tag pointing back to the page itself cause an infinite loop that wastes crawl budget? I assume that search engine robots crawl the whole page and then the canonical

@Connie744

Posted in: #RelCanonical #WebCrawlers

I assume that search engine robots crawl the whole page and then the canonical tag because pages that are self canonicalized still get indexed and their contents are still parsed.

However, does a self canonical link cause an infinite loop wasting crawl budget?

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Connie744

1 Comments

Sorted by latest first Latest Oldest Best

 

@Ann8826881

Bots don’t¹ blindly follow all links on a page.

A bot remembers the URLs it has crawled and the URL it is currently crawling. If it finds a link to one of those URLs again (something which happens all the time -- think of the site navigation, for example), there’s typically no need to crawl it again for some time.



¹ Well, technically they could, of course. And there are likely some peculiar/bugged bots out there that do this. But it would not only waste your resources, also the bot’s. So bot providers have a reason to make their bots "smart": saving costs.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme