: 404 Hits from "PHPCrawl" on WP site Forgive me if this is in the wrong area - I thought considering the topic, it belonged here (as opposed to in WP). Correct me or move it if I am wrong.

@Gretchen104

Is it possible one of the plugins could have caused these entries?


Highly unlikely unless one of your plugins is attempting to index the site. But the plugins have access to the wp-posts table so there isn't any need to spider via the front end.


Is it automated from some search engine?


Potentially, but not one of the major ones as those Spiders are easily and readily identifiable in your logs. I can't think of any legitimate search engine that would use PHPCrawl. You should start tracing the IP addresses that are causing the 404 errors.


Is it something I should be worried about hurting the site or its SEO?


There's not really enough information to answer this definitively. Someone is crawling your site via a script but there's no way to know why or to what purpose they will use the data. Should you be worried? Probably not...this kind of stuff happens all the time. If the IP address proves to be from an area that you are not concerned about, add a deny line to your .htaccess file and block them. Will it affect SEO? Again, most likely not. It is remotely possible that someone is spidering pages to create a spammy page of links but careful checking of your analytics will reveal any weird backlinks and you can then handle those easily enough.

tl;dr

The situation bears watching but is unlikely to be a problem.

Vote Up Vote Down