: Robots.txt problem in subdirectory website (locale) and "/" character We have two sites mainsite.com and mainsite.com/de where /de shows the german site. We rely on 1 robots.txt file. It contains
We have two sites mainsite.com and mainsite.com/de where /de shows the german site.
We rely on 1 robots.txt file. It contains all the disallow links for mainsite.com that we set-up and copied it to mainsite.com/de.
Disallow: /media/sales/
Disallow: /media/tmp/
Question: doesn't the first slash (/) point to the root? How should we set up this robots to work also on the subdirectory?
More posts by @Ravi8258870
1 Comments
Sorted by latest first Latest Oldest Best
First of all /de is not a sub-domain- it is a sub-directory. A sub-domain would be something like de.example.com.
And yes. You will need /de/media/sales/ and /de/media/tmp/. You will always be working from the root directory.
Lastly, the robots.txt file must be in the root directory. You cannot create a robots.txt in the /de directory. It will never be seen. So your robots.txt file in the root will reference:
/media/sales/
/media/tmp/
/de/media/sales/
/de/media/tmp/
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2025 All Rights reserved.