: Robots.txt being ignored - wordpress install I just took over maintaining a website and noticed that Google gives "A description for this result is not available because of this site's robots.txt"

@Annie201

Posted in: #RobotsTxt #Wordpress

I just took over maintaining a website and noticed that Google gives "A description for this result is not available because of this site's robots.txt"

So I went to look at the site's robots.txt file and it shows me the following:

User-agent: *
Disallow: /


Which is weird because there wasn't a robots.txt file in the root. So I created one. That file won't display. So I did some research and saw that I should check Settings >> Reading to see if Search Engine Visibility was checked. So I unchecked it (as suggested numerous places). I then went to the site's domain and manually entered the address (www.eonclinics.com/robots.txt) and the bad robots.txt content is still being read, which is weird because I actually have a robots.txt file that looks nothing like the bad one.

I decided to see if it was just me so I renamed the robots.txt file to robot.txt to see if I could see the info inside and sure enough I could. But if I went to "robots.txt" then I would still see the bad content.

So it's as if something is saying "screw you, use this robots.txt file and ignore anything named "robots.txt" in the root."

I thought maybe it was a matter of caching at first, so I cleared my cache and loaded up Google Webmaster tools just in case. Nope - still same bad robots.txt

Yoast is installed and I turned off sitemaps and turned them back on (I saw that could be a trick but it was a 4 year old post) but no avail. There's no robots.txt plugin or google sitemap plugin installed.

Have tried to turn off all plugins but no go.

Hosting is the Deluxe plan from GoDaddy (not my choice) and theme used is "Total" (not my choice). I am stuck with these choices.

If I use www.eonclinics.com/robots.txt?nopass=1 then I can see the correct robots.txt file. If I use www.eonclinics.com/robots.txt/ then I get the edited robots.txt that is generated in funcitons.php

Has anyone seen a problem like this? Any suggestions? All info I keep coming across is 3 - 5 years old.

Oh - the domain is eonclinics.com.

Vote Up Vote Down

Login to post a comment!