Mobile app version of vmapp.org
Login or Join
Dunderdale272

: Prevent Advert Images Appearing in Search Spider Crawls While testing out how web pages on my site appear to search engine spiders using http://www.browseo.net/ I noticed that advert images appear

@Dunderdale272

Posted in: #Advertising #WebCrawlers

While testing out how web pages on my site appear to search engine spiders using www.browseo.net/ I noticed that advert images appear in the view. However, when I test a site like guardian.co.uk using this same service their main adverts don't show. It seems they are treating advert images and their normal website images differently.

What is the best (penguin friendly) way to achieve this outcome?

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Dunderdale272

1 Comments

Sorted by latest first Latest Oldest Best

 

@Welton855

One, this depends on how the advertisements are generated. Most crawlers understand little or no Javascript, so if you just have the ads Javascript generated, you are done. This is the base for ads from most ad delivery networks, including Google AdSense.

The other is simply to do user-agent detection and deliver ads to user-agents you want. In this case, Google says you must include Vary: User-Agent in your response headers to let then know that you are changing your response according to user-agents.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme