: How to fix Adsense error: "Our crawler was unable to access the pages" when those pages had been removed and are 404 not found? I have a website and we recently changed content management

I have a website and we recently changed content management systems. With the new system came new URLs/hierarchy. Google Webmaster Tools doesn't have any issues with the site and I get no crawler errors. But AdSense seems to still be looking for URLs on my old site and I get tons of "Page not found" errors.

What can be done to get Adsense to crawl the new sitemap?

This is on the crawler errors page. It says "Blocked URLs" and then to the side of the URL it lists "Page not found."

It explains: "Our crawler was unable to access the pages listed in the ’Blocked URLs’ column to determine the content and display ads. Click the ’plus’ icon beside each domain to see an expanded list of affected URLs. When our crawler can’t access your content we won’t show ads, resulting in lower revenue and coverage. Follow the link in the ’Error’ column to learn how to correct these errors."

I just want it to forget these pages exist, because they don't anymore.

10.01% popularity Vote Up Vote Down

: In Google Analytics, see when custom events happen during the same pageview? Is it possible to get a report that shows custom event #1 occurred during the same pageview as custom event #2?

@Pierce454

Posted in: #Analytics #AnalyticsEvents #GoogleAnalytics #PageViews

1 Comments

: When does a .io domain drop and become available once it has expired There is a .io domain name that has expired and I would like to purchase it. I understand with top level generic domains

@Pierce454

Posted in: #DomainRegistrar #DomainRegistration #Domains

1 Comments

: How to get bid simulation reports from AdWords My Client Center? According to this article one can download bid simulation reports on the My Client Center (MCC) level. Unfortunately, I don't

@Pierce454

Posted in: #Bidding #GoogleAdwords #Keywords #Reporting

0 Comments

: Should I remove old links after creating 301 redirect? Let’s say I have a URL like example.com/non-seo-optimized-link. So I create a new route and a 301 redirect to example.com/great-optimized-link.

@Pierce454

Posted in: #301Redirect #Links #Seo

1 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Heady270

AdSense uses a crawler called Mediapartners-Google. It doesn't crawl your entire site using the sitemap. Rather, when a user requests a page with ads on it, it crawls that URL on the fly.

If you are no longer serving pages with ads on those URLs, the AdSense bot will stop crawling those URLs without any intervention from you.

My guess is that when you took the pages down, there were a few users that accessed cached versions of the pages. That triggered the crawl and the error. I would expect the errors to disappear within a couple weeks.

10% popularity Vote Up Vote Down

Feed

: How to fix Adsense error: "Our crawler was unable to access the pages" when those pages had been removed and are 404 not found? I have a website and we recently changed content management

More posts by @Pierce454

: In Google Analytics, see when custom events happen during the same pageview? Is it possible to get a report that shows custom event #1 occurred during the same pageview as custom event #2?

: When does a .io domain drop and become available once it has expired There is a .io domain name that has expired and I would like to purchase it. I understand with top level generic domains

: How to get bid simulation reports from AdWords My Client Center? According to this article one can download bid simulation reports on the My Client Center (MCC) level. Unfortunately, I don't

: Should I remove old links after creating 301 redirect? Let’s say I have a URL like example.com/non-seo-optimized-link. So I create a new route and a 301 redirect to example.com/great-optimized-link.

Login to post a comment!

1 Comments

Back to top | Use Dark Theme