: Help with robots meta tag and X-Robots-Tag I have two questions about robots meta tag and X-Robots-Tag: If a page has both X-Robots-Tag HTTP header and robots meta tag, can this cause problems?

@Ann8826881

If a page has both X-Robots-Tag HTTP header and robots meta tag, can this cause problems?


This should not cause "problems". Providing you make sure to provide the same values for both. eg. noindex, as in your example.

For search engines that support the X-Robots-Tag HTTP response header, then this should always take priority. Where there is an equivalent HTTP response header for an in-page meta element, the HTTP header always wins.


Which one is better? X-Robots-Tag HTTP header or robots meta tag?


Well, you can't use the robots meta tag on non-HTML resources, such as PDFs and images etc. This is primarily why the X-Robots-Tag header was invented.

But whilst all the main search engines support the X-Robots-Tag header, it probably doesn't have as wide support as the robots meta tag, which has been around a lot longer.


On a frame redirect/masked page, does it count the inner robots tags?


Yes. The document contained inside the frame is an entirely separate request and consequently, it's fetched just like any other single request.


Does it count the meta tags of example1.com/masked-url or example2.com/original-url?


Both. They are two separate requests.

Vote Up Vote Down