: Robots.txt, how effective is it and how long does it take? We recently updated the site to a single page site using jQuery to slide between "pages". So we now have only index.php. When you
We recently updated the site to a single page site using jQuery to slide between "pages".
So we now have only index.php.
When you search the company on engines such as Google, you get the site and a listing of its sub pages which now lead to outdated pages.
Our plan doesn't allow us to edit the .htaccess and the old pages are .html docs so I cannot use PHP redirects either.
So if I put in place a robots.txt telling the engines to not crawl beyond index.php, how effective will this be in preventing/removing crawled sub pages.
And rough guess, how long before the search engines would update?
More posts by @Becky754
2 Comments
Sorted by latest first Latest Oldest Best
robot.txt is text file which just includes allow and disallow rules for any particular page of website. If you need to remove all pages from Search engine then you need to use Google web master tool. There is not particular time period need to remove website(updation process of search engine) from Google but approx 5 to 15 days needs to remove it from Google.
The important information about you website is that if you have removed website from Google then you can recover it in 30 days.
Having a robots.txt to tell the search engine not to crawl certain pages should work but there are no guarantees because search engines do what they want. You could also try creating a removal request in Webmaster Tools. The delay for search engine updates vary from a day to months and this also depends on the search engine (I have heard Bing is the slowest).
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.