: What's wrong with my robots.txt file to block Google? I have a website and I don't want this website to be visible on Google. So I deployed a robots.txt file. But the homepage of the website

@Courtney195

Use Disallow: / to block your entire domain from being crawled by google bots.

Vote Up Vote Down

@Bethany197

If you don't want to allowed to crawl your website on Google search result, then use this robots.txt

User-agent: *
Disallow: /


These will block all directories.

But if many website links to your website, then Google will start displaying this snippet in future.



Now if you don't want to index that website competely, then use noindex meta-tags/HTTP headers. That meta tags simply no index your all pages, but it is allowed to crawl. So if your main concern is about to, not visible in search result, then I highly recommended to use noindex tags.

But don't use it both, because when the site is blocked by robots.txt then Google will not going to see your meta-tags/http headers.

Vote Up Vote Down