: How should I submit these urls to Google Webmaster tools On my site you can do a search for a an artist, this gives a URL of the form http://example.com/artistid, this goes to my application

@Si4351233

The first aspect, sitemaps, which was touched on in the comments, will only work with URL's in the same domain, in other words the example.com sitemap should only contain links in the form of example.com/artistid then in the artists.example.com sitemap it should contain the links for artists.example.com, now since the only content that has already been generated is hosted in the S3 bucket and not on the application it would be safe to add the S3 bucket pages to the Google index directly. This could even be automated by re-generating the sitemap.xml file first, then using the Search Console API do a PUT request to www.googleapis.com/webmasters/v3/sites/siteUrl/sitemaps/feedpath where siteUrl is the root domain including the protocol such as www.example.com/ and feedpath is the full URL of the sitemap file such as www.example.com/sitemap.xml. As long as you are updating the lastmod field for the generated page to be the current date and any other pages that have been updated have their lastmod field updated to the right date Google will download a copy of the new sitemap file, and list all the new or modified pages for crawling.

Vote Up Vote Down