Mobile app version of vmapp.org
Login or Join
Angela700

: How do I do an htaccess rewrite on an internal URL without Google crawl giving me a soft 404 error? Here is my issue. I have a rewrite rule in my htaccess like this: RewriteRule ^(news|review|feature|editorial|podcast)/

@Angela700

Posted in: #CrawlErrors #Htaccess #UrlRewriting

Here is my issue. I have a rewrite rule in my htaccess like this:

RewriteRule ^(news|review|feature|editorial|podcast)/([0-9]+)/(.*)$ /article.php?id= [L]


Basically what it does is take in a URL like this:
www.example.com/feature/13889/negative-world-2017-goty-awards

And bring the user to this page:
www.example.com/article.php?id=13889

However, within the last year or so Google seemed to decide it doesn't like this anymore, and will return soft 404 errors when it crawls any of those original URLs. ("The target URL doesn't exist, but your server is not returning a 404 (file not found) error.")

Whether because of this or something else, a lot of my content is no longer indexed on Google, which led to a massive drop in hits for my site.

While reading up on this, I read something that said 301 redirects would solve this. So I switched the line to this in htaccess:

RewriteRule ^(news|review|feature|editorial|podcast)/([0-9]+)/(.*)$ /article.php?id= [L,R=301]


Should that solve the issue, or is there more that I have to do?! I'm a little worried because when I do "fetch as Google" it just says "redirected".

I guess my other question would be, would this explain why this stuff isn't being indexed anymore, or is there probably some other issue I should be looking into?

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Angela700

1 Comments

Sorted by latest first Latest Oldest Best

 

@Jamie184

I can't see how your original internal rewrite would have anything to do with the soft-404 reported by Google. Google is not aware of any rewrite - this is entirely hidden from users, bots and Google.

And changing this to a 301 redirect is only going to make matters a whole lot worse I suspect. Presumably, you are linking to the "pretty" URL (ie. /feature/13889/negative-world-2017-goty-awards)?

A soft-404 is generally reported when Google is served a page that looks like a "Not Found" error, or the page is very thin on content, and which responds with a 200 OK status. From the Google help docs on Soft 404 errors:


A soft 404 means that a URL on your site returns a page telling the user that the page does not exist and also a 200-level (success) code to the browser. (In some cases, instead of a "not found" page, it might be a page with little or no usable content--for example, a sparsely populated or empty page.)


So, whether you serve the page from the "pretty" URL /feature/13889/negative-world-2017-goty-awards or the "ugly" URL /article.php?id=13889 (via a 301 redirect or direct link) then it's still the same page and so still the same soft-404.


would this explain why this stuff isn't being indexed anymore


Google is unlikely going to index a page that it considers to be a (soft) 404.


I'm a little worried because when I do "fetch as Google" it just says "redirected".


I would remove the redirect and redo the "fetch as Google" check. What is Google seeing? Maybe it's getting a different response than what users are seeing?

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme