: In WordPress, should I "Allow search engines to index this site" during development? I am going to create a web page, and I will do it online. Should I choose the option: "Allow search engines

@Angela700

It really depends on whether you care if your site is viewed when "under construction".
Usually it's best to wait until after the site is complete to index.

Vote Up Vote Down

@Heady270

Its best to block allow while starting a new website,

If you are rebuilding an existing site then you can't do disallow options,
The main reason is while you creating new site you should have some dummy content on it,
The Search Engine will crawl the site they indexed your dummy content is a bad idea.

So my suggestion is to disallow Search Engine Crawl from robots.txt until you site have proper content or launched.

Also while developing the site you can access the site without domain name like direct ip its also good practice.

Hope it make sense..

Vote Up Vote Down

@BetL925

Development never really ends. Every active website could have an "under construction" banner on it.

You should allow Googlebot to crawl your site as soon as you have your first piece of content that users would find useful.

Some webmasters even allow Google to crawl a "coming soon" notice that they put up as the only thing on the home page. I'm not a big fan of "coming soon", but I don't know of any good reasons that you shouldn't allow such a page to be crawled by Google.

The only time that you should prevent search spiders from crawling an entire website would be when site has information on it that you don't want to have indexed in the search engines.

Vote Up Vote Down