: Can I include a robots meta tag outside of the head in HTML snippets indeded to be AJAXed? I have a number of files in my site which are not intended for independent viewing, but rather
I have a number of files in my site which are not intended for independent viewing, but rather to be AJAXed into content within the site. They obviously don't meet HTML standards (no body, head, etc.) as independent entities. I would like to prevent search engines from indexing these pages, but do not have access to /robots.txt (which would be much more ideal).
My question is, could I include the following at the top of these partial HTML files and get the desired results?
<meta name="robots" content="noindex, noarchive">
I guess there are two parts to this question.
Will this cause any rendering issues in any browsers?
Will search engines (at least Google & Bing) interpret this as intended?
More posts by @Cody1181609
5 Comments
Sorted by latest first Latest Oldest Best
Unfortunaly - and this isn't documented by google - it works.
I've a client site with a ajax adv that injected a asynchronously into body, and Google deindex that page.
Since you don't have access to robots.txt then .htaccess is also out of the question.
You might consider a HTTP HEADER.
See: developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag
X-Robots-Tag: noindex
PHP code example:
<?php header("X-Robots-Tag: noindex", true); ?>
If those pages are made with a server side language, you can include a verification for a constant or a variable, if that exist, then the file can be loaded, if it doesn't exist, you can redirect to another file or just block it.
It's much better to send a redirection.
You can do that also with the .htaccess file detecting who is requesting the file, for instance, if a bot is requesting the file, and you only want to send it to browsers, then you can block or redirect the bot before it reaches the server side language.
You can put the tag on the top of an HTML document. It will not cause rendering issues, since meta elements are not displayed, and this type of meta does not affect the way the page is rendered. Search engines will process it as in any other HTML document.
HTML specifications do not require body or head tags (except XHTML, but nobody cares that, unless you serve your document with an XML content type). An HTML document always has body and head elements, but their tags are optional. If you have a document containing just
Hello world
and if it is served as text/html, it will be processed as an HTML document containing a body with that text as content and a head element with no content, as if the document contained
<html>
<head>
</head>
<body>
Hello world
</body>
</html>
(This is invalid, since a title element is required, but this does not prevent browsers and search engines as processing the data as HTML.) If you add a meta tag at the start, it will automatically become the content of the head element.
<meta> is intended to be in the <head>. If you inject in into an existing web page, you will end up with invalid markup (after ajax)
But you are drowning in a cup of water :-) If it's intended to be loaded via ajax. Surely you will not have any link <a> to these files. If you don't link to the files, the bots will not index them.
So, you are trying to solve a non existent problem here.
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.