: Google shows the wrong title for my website - robots.txt issue Let's cut to the chase. I have this website all set up and everything, but when I type "lissa mariage" on google it shows

@Moriarity557

Actually, if you want all crawlers to be able to crawl whole your website, you don't even need robots.txt, because if you don't have robots.txt file, crawlers would assume that they are allowed to crawl your website.

Vote Up Vote Down

@Heady270

Your user agent for Google wrong. It should be Googlebot. A user agent of Google is probably not recognized. See Google's "robots.txt" section of their help article Monitor crawling activity and errors.

Vote Up Vote Down

@Shelton105

Sometimes Google crawls webpages before the proper title is setup so you might not have the right title displayed. That however goes away soon after the page is crawled again.

The message you are seeing basically says that this particular page is blocked by your robots.txt. Your former co-worked just blocked all web crawlers from getting to your site.

What you need to do is change your robots.txt to:

User-agent:*
Allow:/


Note: this will allow all crawlers to crawl your entire site. You can limit what is crawled by adding something like this:

User-agent: *
Disallow: */site-section/


Then wait until the site is crawled again or go to your GWT and 'ask' google to crawls your site again. Below is a screen show:



If you want the whole site crawled then just hit the fetch button.

Note: changes might take a few weeks to take place.

Vote Up Vote Down