: If I block a page in robots.txt would I still be able to see the page in my browser? What is the purpose of robots.txt? Is it to prevent bots from crawling pages? And if so, does that

@Mendez628

Yes, you can still access the page through your browser. robots.txt is a suggestion to search engines about pages or directories you would like them not to crawl. Most major search engines such as Google honor these requests, but there is nothing that physically forces them to or guarantees that they or any other search engine will.

Vote Up Vote Down