: What is the "google-xrawler" user agent used for? I'm getting over 100 requests per day from user agent google-xrawler. The requests are all for the same page. Some of the IP addresses are

I'm getting over 100 requests per day from user agent google-xrawler. The requests are all for the same page. Some of the IP addresses are known to be Google IPs, although they don't have valid reverse DNS records.

What is this user agent used for?

10.01% popularity Vote Up Vote Down

: How can I give FTP access to multiple folders in cPanel? I am trying to create a FTP account in cPanel, and give it FTP access to multiple directories. There are a lot of folders, but

@Gonzalez347

Posted in: #Cpanel #Directory #Ftp #Path

1 Comments

: What should be the standard time for a crawler to revisit my site? I've found mostly on the site meta name "revisit-after" content="3 days" and also I'm using this. But the fact is, what should

@Gonzalez347

Posted in: #GoogleSearchConsole #Seo

3 Comments

: Duplicated webpage (HTTP & HTTPS) my webpage is duplicated because of the SSL certificate (http:// example.com and https:// example.com). I want to redirect from http to https. But I have two

@Gonzalez347

Posted in: #DuplicateContent #Https #SecurityCertificate

1 Comments

: Why wouldn't a website apear in Google search results after robots.txt update to clean up hacked site? I have a WordPress website which was hacked couple of days ago. I have tried to add

@Gonzalez347

Posted in: #RobotsTxt #Seo #WebCrawlers #Wordpress

1 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Chiappetta492

Google requests product feed using the user agent "google-xrawler"

10% popularity Vote Up Vote Down

Feed

: What is the "google-xrawler" user agent used for? I'm getting over 100 requests per day from user agent google-xrawler. The requests are all for the same page. Some of the IP addresses are

More posts by @Gonzalez347

: How can I give FTP access to multiple folders in cPanel? I am trying to create a FTP account in cPanel, and give it FTP access to multiple directories. There are a lot of folders, but

: What should be the standard time for a crawler to revisit my site? I've found mostly on the site meta name "revisit-after" content="3 days" and also I'm using this. But the fact is, what should

: Duplicated webpage (HTTP & HTTPS) my webpage is duplicated because of the SSL certificate (http:// example.com and https:// example.com). I want to redirect from http to https. But I have two

: Why wouldn't a website apear in Google search results after robots.txt update to clean up hacked site? I have a WordPress website which was hacked couple of days ago. I have tried to add

Login to post a comment!

1 Comments

Back to top | Use Dark Theme