Mobile app version of vmapp.org
Login or Join
Cooney921

: Do I need to add subdomain sitemap / robots.txt separately? Say I have a domain example.com And it has a subdomain sub.example.com And another subdomain internal.example.com I have some links

@Cooney921

Posted in: #RobotsTxt #Sitemap #Subdomain

Say I have a domain example.com

And it has a subdomain sub.example.com

And another subdomain internal.example.com

I have some links on example.com and sub.example.com for Google to crawl. But I don't want Google to fetch anything on internal.example.com

What should I do in this case?

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Cooney921

1 Comments

Sorted by latest first Latest Oldest Best

 

@Kristi941

Here is link to Google's documentation about the validity of URLs robots.txt as per its location.

Do you have any links from the main domain or the other sub-domain to internal.domain.com?

If not then Google, in general, would not see your sub-domain on its own.
I would still recommend to block it via the robots.txt file if they have never been exposed in past. If, by any chance URLs from internal.example.com have been exposed to Google then use Robots-Meta Tag to declare these pages non-indexable and no-follow.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme