: Ajax Content from Blocked Resource I have a site built in AngularJS. Most of the dynamically-loaded content comes from a Wordpress back-end that is separate from the AngularJS site. In fact,

@Alves908

You should not block resources that are used in creating/rendering the page you want to index to avoid being negative in google's compilation.

But if you are working on domain x and loading something from domain y that you don't want crawler to index. Then you should not block the resources from y, instead on pages from Y, you can add meta noindex, which will let crawler crawl this from domain x, but won't list it in its search page results and would be unreachable for users using organic channel.

This is what i think you want to achieve.

Vote Up Vote Down

@BetL925

Google needs to be able to fetch the content from the WordPress site to be able to render the pages from your Angular.js site. Checking the "Discourage search engines from indexing this site" disallows crawling in the site's robots.txt file. This prevents Googlebot from fetching the content, even when called from another domain.

Instead, you want to allow Googlebot to fetch the content but tell it about the preferred URL where it can be viewed. You can use a meta rel canonical tag on each page of the WordPress site to point Googlebot to the corresponding page on the Angular.js site. For example, wordpress.example.com/some-page might have the tag <link rel="canonical" href="http://angular.example.com/#some-page" />

Vote Up Vote Down