: How to specify priority content for Google to index first when launching a very large website My team will go online with a portal that will have over 1.000.000 products. And I read here following
My team will go online with a portal that will have over 1.000.000 products. And I read here following text:
Prioritize which content you have indexed. With that much data, not
all of it is going to be absolutely vital. Make a strategic decision
as to what content is most important, e.g., it will be most popular,
it has the best chance at ROI, it will be the most useful, etc. and
make sure that that content is indexed first.
Another SEO expert told me to pick out our top 50 categories (including all products) and to put them into our XML Sitemap for Google.
Our site map for users gives already deeplinks into those categories and manufacturers too.
And if we put our top 50 categories into our XML Sitmap, should we:
a) just increase the number of categories as soon as google got almost everything crawled?
b) just take out categories that do not work and put in others that work better? So that Google Bot is focused on our top 50 categories (with approx 250.000 Products)
As far as I know Google indexes everything, as long as there's no no-index set - but a XML Sitemap will help Google to find our important pages faster - is this correct?
More posts by @Radia820
3 Comments
Sorted by latest first Latest Oldest Best
From the GWT you can "Add to Index" upto 500 important URLs in a month and Google will crawl them in a day or two. I guess you can add site portions like sub domains and folders as separate sites and again use 500 URL submission limit for each.
The rest will be picked up by the sitemap and Google will crawl at it's own pace. The other part have already been answered by experts above.
John's answer is very good. I agree that the best way to get your priority content indexed is to feature it prominently on your site.
The other tool that you have at your disposal is robots.txt. You could use robots.txt to prevent Google from crawling large sections of lower priority content when you first launch the site. Later, when the site is better established, you could open up these areas for crawling as well.
a) just increase the number of categories as soon as google got almost everything crawled?
b) just take out categories that do not work and put in others that work better? So that Google Bot is focused on our top 50 categories (with approx 250.000 Products)
Not placing pages in your sitemap doesn't hide them from Google as they can find them by naturally crawling your site. So this won't spoon feed Google the pages you want indexed.
If you want to give to certain categories more weight in Google's eyes, give them more weight on your website. You can do this by linking to them more frequently and prominently in your website. Linking to these categories from your homepage or any other page that is high in the structural hierarchy of your website will give them higher precedence to Google as links from those pages will be stronger then links from deeper inner pages.
As far as I know Google indexes everything, as long as there's no no-index set - but a XML Sitemap will help Google to find our important pages faster - is this correct?
No. Sitemaps tell search engines where to find your content. It does not give weight or priority to what is to be crawled (despite their being a priority flag in the sitemap).
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.