Mobile app version of vmapp.org
Login or Join
Kristi941

: Robots.txt vs. Noindex Tags According to matt cutts, even though we block the page using Robots.txt , it is better using noindex tag. ( Source : https://www.youtube.com/watch?v=KBdEwpRQRD0) If it

@Kristi941

Posted in: #Noindex #RobotsTxt #Seo

According to matt cutts, even though we block the page using Robots.txt , it is better using noindex tag. ( Source :


If it is blocked by Robots.txt, how Google bot read whether it has noindex tag or not? Because as I know if it blocked by Robots.txt, Google will not read any thing in that post.

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Kristi941

1 Comments

Sorted by latest first Latest Oldest Best

 

@Pope3001725

Good catch! If you're blocking a page with robots.txt then crawlers will not able able to read the noindex meta tag. In these cases you should send the x-robots-tag HTTP header either via server-side code or .htaccess.

.htaccess example:

X-Robots-Tag: noindex


Sample PHP code:

header('X-Robots-Tag: noindex');

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme