: How to hide download file from bots? Possible Duplicate: How to restrict the download of all files in a folder? I want to make a private file available for download but not use

@Correia994

First of all you can add a proper robots.txt entry to tell willing bots to ignore and not index your specific directory.

Bots should follow your robots.txt intentions, but some people (or bots) will read them on purpose to get to know what you want to hide.

So you will have to make sure your webserver does not automatically create an index of all files in your directory and create hard-to-guess filenames. This is security by obscurity and bad.

When returning the file you could also think of adding the relatively new X-Robots-Tags as a last resort and add directives like "noindex,noarchive". This way google bot even if it finds the file should not index and cache it. But others not knowing the tag will.

Programmatically you could also write code that returns the file for download only if the accessing client is no bot. Problem here: Bot-detection is a never ending story and could be faked - as almost everything can what comes with the HTTP header.

Maybe you can limit the access by IP address ranges as another measure to limit the range of clients with access or to exclude known bot ip addresses (but those often change).

Vote Up Vote Down