: Web Host / Service to Fetch Batch of URLs I have a list of about one million URLs that I would like fetched and would rather not run this process on my own web server. I am wondering if

I have a list of about one million URLs that I would like fetched and would rather not run this process on my own web server. I am wondering if there are any commercial solutions to this - i.e. any services that will take my list of URLs and return the HTML for each (running it behind proxies, etc., if necessary).

10.01% popularity Vote Up Vote Down

: This is by far an incomplete list, cause ultimately the full list will include what's in core-utils and many other packages. But the list you have, can easily be sumplemented with whats in

@Jennifer507

0 Comments

: Pseudo-external links and SEO I don't know, how to name such a links. Main idea is following: I write a post at a community, which I do not control, and place a link to my web-site:

@Jennifer507

Posted in: #GoogleSearch #Links #Seo #WebCrawlers

3 Comments

: Old Domain Pay-per-click So you know when you go to a domain that say you want to buy, and the only thing that comes up is a bunch of links to other sites, and stuff like that? How do

@Jennifer507

Posted in: #Advertising #Domains

2 Comments

: The best solution I can think of is point an A record in your DNS to the elastic IP from AWS. This has the advantage that it never actually hits your host - it goes straight from the

@Jennifer507

0 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Cody1181609

A Linux virtual private server (some priced at less than /mo) can be used to complete this task with minimal configuration - see the wget man page for instructions if you choose to go this route.

10% popularity Vote Up Vote Down

Feed

: Web Host / Service to Fetch Batch of URLs I have a list of about one million URLs that I would like fetched and would rather not run this process on my own web server. I am wondering if

More posts by @Jennifer507

: This is by far an incomplete list, cause ultimately the full list will include what's in core-utils and many other packages. But the list you have, can easily be sumplemented with whats in

: Pseudo-external links and SEO I don't know, how to name such a links. Main idea is following: I write a post at a community, which I do not control, and place a link to my web-site:

: Old Domain Pay-per-click So you know when you go to a domain that say you want to buy, and the only thing that comes up is a bunch of links to other sites, and stuff like that? How do

: The best solution I can think of is point an A record in your DNS to the elastic IP from AWS. This has the advantage that it never actually hits your host - it goes straight from the

Login to post a comment!

1 Comments

Back to top | Use Dark Theme