: Can google index/crawl random subdomains that aren't linked to from anywhere Me and a friend are having a debate where she claims that every and all subdomains will get crawled/indexed by google

@Berumen354

Nope if you sub domain was not having any high quality content or any link from anywhere (from main domain or from anywhere else) google was not going to index/crawl your random subdomains.

Vote Up Vote Down

@Angela700

Yes, Google can do that and it is best to assume that anything that is publicly available on the internet may be indexed by Google. Linked to or not.

Of course, if you don't link to it the chances of it being indexed go way down. However, Google uses a multitude of tools to gather URLs for indexing. Recently there was a news item about Dropbox links that had been shared (thus becoming publicly available) being indexed by Google because people clicked on links in the documents or put their URLs in Google search box.

It doesn't really matter how Google finds the link. The point is that it may.

So the bottom line is that if you don't want something to wind up in Google you must put a robots.txt file in place to keep Google out. Relying on obscurity is not advised.

Vote Up Vote Down

@Harper822

Think of the linking as a chain reaction. Google won't link to domains if it has no way of accessing it or even finding it. If a friend advertises your URL on a popular forum site that Google always indexes, then there's a chance Google will scan your URL and possibly index it, thinking the link may be part of the site.


Is the fact I've installed wordpress on it making my installation "ping search engines"? How does that work?


I doubt content management systems will randomly ping search engines, but you can always check the source code and see if any code contains commands for opening remote URL's. Examples of such code in PHP (which is what wordpress uses) might include (in no particular order):

$data=file_get_contents("http://www.searchengine.com/submittoengine/data.cgi?whatever=whatever");

$remote=fopen("http://www.remote.com/upload.cgi?website=bla.com");


Or even curl functions which may include:

curl_exec($webdata);


A good way to see how wordpress behaves network wise is to create your own LAMP/WAMP setup. This means use Linux or Windows and install Apache, MySQL and PHP on one computer then disconnect from your actual internet and setup Apache so that you can access content when you type in one of the following URLs:
127.0.0.1/ http://localhost/


When apache is installed correctly for the first time regardless of the status of your real internet connection, you should see something like "It works" or something other than a "could not connect to remote server" type of message. Next, install wordpress and then see if it complains about the internet or pinging, etc. I bet it wouldn't.

Everything is best done by experimentation.

Vote Up Vote Down