Mobile app version of vmapp.org
Login or Join
Steve110

: Can I use sitemap.xml to redirect bots to seo specific urls I apologize for the title, but I can't really explain it . Here's what I am asking Can I have seo specific urls in my sitemap.xml

@Steve110

Posted in: #CanonicalUrl #Seo #SinglePageApplication #Sitemap

I apologize for the title, but I can't really explain it . Here's what I am asking

Can I have seo specific urls in my sitemap.xml that will have canonical links to the actual urls.

E.g: Sitemap xml has a url of /foo/bar?seo=true and the content has a canonical link to /foo/bar

The reason we are considering this option:

We have a javascript SPA, that needs to be pre-rendered for bots. In order to avoid the overhead of checking on every request, we thought we can use sitemap.xml to redirect bots to a specific seo url, which will go through a pre-redenderer (e.g: phantomjs) and return the actual content, which includes a canonical link to the actual url.

10.03% popularity Vote Up Vote Down


Login to follow query

More posts by @Steve110

3 Comments

Sorted by latest first Latest Oldest Best

 

@Gloria169

Have you tried using an htaccess rule to apply additional $_GET params when your website is requested by Google?

From this Stack Overflow answer:

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^(.*).google.(.*) [NC]
RewriteRule ^(.*)$ www.my-site.it/ [L,R]

10% popularity Vote Up Vote Down


 

@Margaret670

You're fine to use parameter(?) in your XML sitemap, like ?seo=trueor similar words.

You can add same parameter to above question and you will see it display the same page,
webmasters.stackexchange.com/questions/102669/can-i-use-sitemap-xml-to-redirect-bots-to-seo-specific-urls?seo=true

When Googlebot see your parameter URL from sitemap, and start crawling, it sees canonical link tag, which point to the non parameter URL, so they will assign you're prefer to index that webpage(/foo/bar), not the current one(/foo/bar?seo=true).

Feel free to use parameter URL in sitemap, but don't use it on internal link strcuture, I have seen many of webpages that index with parameter and it is just because Google priority the link structure compare to canonical link tag.

10% popularity Vote Up Vote Down


 

@Nimeshi995

You could, but you won't get the guarantee that it won't break users experience, leading a user to reach your website with ?seo=true appended to his request.

To make an example, Google specify that


We attempt to respect this, but cannot guarantee this in all cases.


meaning you should then check that they're actual users rather than bots.
To be fair, IMHO, I won't expect that to happen that often, thus could effectively reduce the overhead due to a reduced number of requests to require special handling.

Furthermore you're not sure that a bot reaches your website via a link (which may be external to your website) rather than using the sitemap, and in that case you'll not serve the pre-rendered page. Although you have a sitemap the crawler algorithm may decide to crawl the page immediately without looking at the sitemap (at least at that moment).

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme