Mobile app version of vmapp.org
Login or Join
Ravi8258870

: Robots.txt problem in subdirectory website (locale) and "/" character We have two sites mainsite.com and mainsite.com/de where /de shows the german site. We rely on 1 robots.txt file. It contains

@Ravi8258870

Posted in: #RobotsTxt

We have two sites mainsite.com and mainsite.com/de where /de shows the german site.

We rely on 1 robots.txt file. It contains all the disallow links for mainsite.com that we set-up and copied it to mainsite.com/de.

Disallow: /media/sales/
Disallow: /media/tmp/


Question: doesn't the first slash (/) point to the root? How should we set up this robots to work also on the subdirectory?

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Ravi8258870

1 Comments

Sorted by latest first Latest Oldest Best

 

@Jamie184

First of all /de is not a sub-domain- it is a sub-directory. A sub-domain would be something like de.example.com.

And yes. You will need /de/media/sales/ and /de/media/tmp/. You will always be working from the root directory.

Lastly, the robots.txt file must be in the root directory. You cannot create a robots.txt in the /de directory. It will never be seen. So your robots.txt file in the root will reference:

/media/sales/
/media/tmp/
/de/media/sales/
/de/media/tmp/

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme