: It depends. All robots.txt does is tell a crawler what content you want—or don’t want—indexed. But that just communicates to the crawler. How the owner of the crawler handles the indexing—and

@Lee4591628

It depends. All robots.txt does is tell a crawler what content you want—or don’t want—indexed. But that just communicates to the crawler. How the owner of the crawler handles the indexing—and cleanup—of the content depends on their internal processes. Meaning, they can take as long as they want to handle that aspect of the process.

In practical terms I have seen results change as quickly as a few days to one full month. This unpredictability is why the whole concept of SEO (Search Engine Optimization) is based on web site owners/managers taking an “ounce of prevention versus pound of cure” approach to website management. Meaning the only thing you can control is your code. The rest of it is out of your hands so you need to present your pages as cleanly as possible.

Vote Up Vote Down

Login to post a comment!