: How to block some CSS files from Googlebot using robots.txt while allowing other files in the same directory? Hi in my Drupal website site I have a few directories like: /sites/modules/ Now

@Deb1703797

Now I only want googlebot or other bots to index or allow say file user.css


a CSS file? it doesn't offer value to an end user.


I don't want Googlebot to index the file lightbox.css in the same directory. How do I edit robots.txt to achieve the above?


Do nothing.

Search engines are only interested in high-quality content which means text. Such text is generally located in html files (or even processed by accessing script files such as php files).

I'm assuming the CSS files are required to make your page function properly. These need to be accessible to the public so that search engines can access the files and render the pages correctly to better understand them. These days, page formatting is often done in CSS and if the CSS file isn't accessible then the page will look almost like a mess at best.

To prove my theory, register an account with Google search console and use the fetch as google tool to try to load the HTML file that requires the CSS files that you are "trying to not make google index" and you can see that if the files aren't accessible then the page can't load as intended.

Vote Up Vote Down

@Mendez628

You can use following to disallow in robots.txt:

disallow: */lightbox.css

Vote Up Vote Down