: Does Googlebot crawl items that look like URLs in HTML5 data-* attributes? Google is discovering some URLs in GWT which I'm surprised it could discover. The only place these are linked from
Google is discovering some URLs in GWT which I'm surprised it could discover. The only place these are linked from are as follows:
<li class="multiselect__item" data-key="filter__brand--examplebrand2" data-name="examplebrand2" data-count="92" data-url="/example-category/examplebrand1--examplebrand2/" title="examplebrand2"> <input type="checkbox" id="checkboxfilter__brand--examplebrand2"> <label for="checkboxfilter__brand--examplebrand2"><span></span>examplebrand2<i class="multiselect__item-count"> (92)</i></label></li>
The code below being the interesting bit:
data-url="/example-category/examplebrand1--examplebrand2/"
GWT is reporting that these URLs are linked from a page on the site, and not externally. My question, should Google be able to crawl these URLs?
More posts by @BetL925
1 Comments
Sorted by latest first Latest Oldest Best
Googlebot uses heuristics to pick out anything from the page that looks like it might be a URL. It can crawl:
URLs in SELECT option values
URLs in data attributes
URLs in JavaScript variables
In my experience, just having a slash (/) in a string is enough for Googlbot to think it may be a URL. The fact that you name you attribute data-URL may also contribute to Googlebot recognizing it.
As far as I know, Googlebot only uses these "links" for content discovery. I don't believe that it passes link juice across them the same way that it does for normal a href links.
Unfortunately, if Googlebot gets it "wrong" and crawls something that isn't a URL, it will still report it in your site errors in Google Webmaster Tools. In my opinion, it shouldn't do that when it used a heuristic to try to find something that looks like a URL.
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.