Mobile app version of vmapp.org
Login or Join
Cugini213

: Can't find 404's that are listed I've been asked to look at the "Not Found" tab in Google Webmaster Tools, to try and eliminate as many of the thousands of 404's not listed. There are several

@Cugini213

Posted in: #CrawlErrors #GoogleSearchConsole #Sitemap

I've been asked to look at the "Not Found" tab in Google Webmaster Tools, to try and eliminate as many of the thousands of 404's not listed. There are several reasons for these, but this post will focus on one type. 404's which cannot be located within our site.

Our WordPress installation will redirect the 404 page to the same page, but with Google Search parameters in the querystring to display a Google search to help the users find what they are looking for.

So if a user goes to:
ourdomain.com/pagedoesntexist.php
they will get redirected to
ourdomain.com/pagedoesntexist.php?cx=987363876 (etc. this is the Google Search Info)

Now in our "Not Found" tab, it lists the URL with the Google Search code as the page, but lists the first page (with no querystring) as the referring URL.

Where I get lost is here. If I do a complete site crawl of our websites, or search the sitemaps for "www.ourdomain.com/pagedoesntexist.php", I cannot find a reference to it anywhere.

So I am confused as to why Google doesn't detect the original page as the 404, since I am guessing WordPress would be handling 404's the way Google expects them to. I am also confused as to how I can get rid of a 404 that I don't have a proper referrer to, as well as why a complete crawl wouldn't reveal the same 404's that Google finds.

When I crawl the site with Xenu, out of 35,000 pages, it finds approximately 50 404's, most of which are GIF/PNG/JS/CSS glitches and errors.

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Cugini213

1 Comments

Sorted by latest first Latest Oldest Best

 

@Heady270

You can crawl your website with Site Visualizer software to find 404 pages, find 'pagedoesntexist.php', as well as view all redirects within a site, if any.

After you've downloaded and installed Standard or Pro version (unlimited 30-day trial), click Project -> New, enter your website URL, then uncheck some options as follows:



Click OK, then Start button. After the crawling is complete, double-click Internal Links report to show all internal links of your site, then click Find command of the report's table context-menu:



Type 'pagedoesntexist' text and click Find Next button. To faster the search, you can uncheck all fields except From URL and To URL. This allows you to find pages that reference to 'pagedoesntexist.php' page, and vice versa.

You can also export the data to an MS Excel sheet or similar. Use Copy Rows commands from the table's context menu.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme