: Sitemap.xml for a website with forum We have a website dedicated to our ActiveX and .NET components that has been working for more than 12 years. The site consists of 2 parts: the static part

@Ann8826881

It might surprise people exactly how old-school search engines are. Of course they have adapted well to all the new technologies such as AJAX and what not. However, much of what has always existed in search engines still exists and is the preferred method.

As far as sitemaps, they are intended only to inform search engines of what pages your site may have. Generally speaking, sitemaps benefit those sites that are very large, cannot or do not link all pages directly, have a barrier such as a pay-wall or login, or where not all pages can be found easily via the linking method used.

Search engines generally prefer to spider any site the old fashioned way. However, if a sitemap is supplied, search engines will compare the sitemap to the pages they can already capture without a sitemap. If the two lists are the same or end up being the same, search engines will prefer to spider and index any site by following links. Part of the reason for this is rather simple. Links signal importance and sitemaps remove this signal. It is quite common that other than reading the sitemap for comparison, search engines will ignore a sitemap. One common exception is for very large sites.

If your dynamic content is available via normal linking, then a sitemap may not be necessary. There does not seem to be a consensus as to how large a site has to be for a site map to be convenient. It never hurts to supply a sitemap. But for a site (and I am assuming something here) about 100k or 200k pages or so, the sitemap may not be necessary. I did not supply a sitemap for a site until it reached about 300+k pages and even then it was not necessary until I stopped linking to all the pages directly.

I wrote my own code for this since I wrote my own CMS. Many CMS and Blogging software will provide options for creating sitemaps. I am not familiar with these options. Sorry. There are also other external spiders that will generate a sitemap for your site. The only one I am familiar with is GSiteCrawler. I used it only once or twice years ago and it worked very well.

If you are familiar with writing your own code, you can follow the protocol found here: www.sitemaps.org/protocol.html. This is a very simple thing to do. I used this site originally to create my own sitemap.

There is no search engine metric for a sitemap that gives an advantage in the search engine result page(s) SERPs. There may be a residual advantage if your site is difficult to crawl.

Again, it never hurts to supply a sitemap. Especially if you are not sure there is an advantage for the search engines. There is always a gray area if your site is somewhat large or you are not sure that search engines can follow your links okay. I say go for it just to be safe. Webmasters will worry about providing a sitemap that often is not necessary. Especially if the site is moderate in size or smaller and can be crawled easily. Again, if it helps you sleep at night, I say go for it. Otherwise, my general advice is not to worry too much unless your site is very large or cannot be crawled easily.

Vote Up Vote Down