: This does not seem possible, although it may depend on your particular webserver, but I strongly doubt this. As far as I know, www.example.com and www.example.com/ are the same thing. PHP certainly

@Odierno851

This does not seem possible, although it may depend on your particular webserver, but I strongly doubt this.

As far as I know, example.com and example.com/ are the same thing. PHP certainly treats then that way too (if you use parse_url() on both you get exactly the same results).

The first rule of your robots.txt allows the root of the site to be accessed (slash or no slash) and the second disallows everything since / technically begins all paths.

What can make things even more confusing is that some browsers remove the final slash and others add them in the location bar. They do that because the send the same request in both cases, basically asking for / on example.com.

Vote Up Vote Down

Login to post a comment!