: Why important/big sites don't include a sitemap? I've been looking at most important sites and none of them have a sitemap.xml defined. My list includes: Amazon Ebay Yahoo Microsoft ... and others

@Eichhorn148

Just checking sitemap.xml is not enough. There are several reasons that that check might fail even though the site has a sitemap.


Did you check sitemap.xml.gz? Google supports gzip of sitemaps. Large sites with large sitemaps are likely to take advantage of this feature.
You can specify the name of the sitemap file in robots.txt. It doesn't have to be sitemap.xml. A robots.txt can have a line like this that says where to get the sitemap. Sitemap: example.com/mysitemapfile.xml It is also possible to create a sitemap file at a "secret" URL. You can tell Google about this URL through webmaster tools. That way Google will get your sitemap, but it won't be easily accessible to the public.




Looking in robots.txt, many of the sites you list do have sitemaps.

Amazon

Sitemap: www.amazon.com/sitemap-manual-index.xml Sitemap: www.amazon.com/sitemap_vendor_videos_us.xml Sitemap: www.amazon.com/sitemap_vod_index.xml Sitemap: www.amazon.com/sitemaps.f3053414d236e84.SitemapIndex_0.xml.gz Sitemap: www.amazon.com/sitemaps.1946f6b8171de60.SitemapIndex_0.xml.gz Sitemap: www.amazon.com/sitemaps.bbb7d657c7e29fa.SitemapIndex_0.xml.gz Sitemap: www.amazon.com/sitemaps.11aafed315ee654.SitemapIndex_0.xml.gz Sitemap: www.amazon.com/sitemaps.c21f969b5f03d33.SitemapIndex_0.xml.gz

Ebay

Sitemap: www.ebay.com/lst/PDP_US_main_index.xml Sitemap: www.ebay.com/lst/PDP_US_incremental_index.xml Sitemap: www.ebay.com/lst/VIP_US_index.xml Sitemap: www.ebay.com/lst/SRP_US_index.xml

Microsoft

...
Sitemap: www.microsoft.com/en-us/dynamics/sitemap.xml Sitemap: www.microsoft.com/en-us/kinectforwindows/sitemap.xml Sitemap: www.microsoft.com/en-us/sqlazure/sitemap.xml Sitemap: www.microsoft.com/en-us/cloud/sitemap.xml Sitemap: www.microsoft.com/en-us/server-cloud/sitemap.xml ...

Vote Up Vote Down