: Robots.txt vs. Noindex Tags According to matt cutts, even though we block the page using Robots.txt , it is better using noindex tag. ( Source : https://www.youtube.com/watch?v=KBdEwpRQRD0) If it
According to matt cutts, even though we block the page using Robots.txt , it is better using noindex tag. ( Source :
If it is blocked by Robots.txt, how Google bot read whether it has noindex tag or not? Because as I know if it blocked by Robots.txt, Google will not read any thing in that post.
More posts by @Kristi941
1 Comments
Sorted by latest first Latest Oldest Best
Good catch! If you're blocking a page with robots.txt then crawlers will not able able to read the noindex meta tag. In these cases you should send the x-robots-tag HTTP header either via server-side code or .htaccess.
.htaccess example:
X-Robots-Tag: noindex
Sample PHP code:
header('X-Robots-Tag: noindex');
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.