: Remove dev URL of website from Google Search I have the following website: However somewhere along the way Google seems to have decided to index my old development URL which doesn't exist

@Berumen354

I've seen this happen many times in the past. Based on the limited information provided, it seems as though your issue is not with the dev url, but rather with your old dev hosting account. You see - google would not be indexing the pages if they did not exist on the server.


You could always delete them, but that would prevent future use of your dev server if you want to rework some functionality & then go live.
This could have been avoided altogether by using a noindex,nofollow meta tag so that google would have never indexed your dev site in the first place. The tag is a meta tag on your root url, along with every other page on your site. This setup appears as follows:




However, it is of the utmost importance that you remember to remove the noindex,nofollow tag prior to going live with your dev site - otherwise you will risk deindexing your main url in Google Search, which is not a fun experience.

lastly, I suggest trying to locate the server, and using an .htaccess (if on apache) to prevent indexation and crawling of the dev website. The code would look as follows, and you can simply copy/paste it into the .htaccess (if you don't have an .htaccess, then make one):

Header set X-Robots-Tag "noindex, nofollow"

With the above code implemented, the dev pages will no longer appear in search results, and your problem, from my limited understanding, will be solved.

Vote Up Vote Down

@Rambettina238

Yes blocking any URL with robots.txt is a way, but you can also create a Google Search Console property for development copy of your website and remove the URL permanently.

Vote Up Vote Down

@Connie744

You can ask Google not to crawl in couple ways:

Method 1: Using robots.txt file. Create a robots.txt with code below and place in your dev server root folder.


User-agent: *

Disallow: /


Method 2:

The second method, just restrict the access using username and password. If you are using cPanel hosting, you can easily do this, and same in other hosting too. Meaning when you will fetch your dev server it should ask you to login.

Vote Up Vote Down