Mobile app version of vmapp.org
Login or Join
Shanna517

: Does operating a web crawler/scraper use up my website bandwidth? I would like to operate a website crawler/scraper from my web server (doing so ethically of course). What I am wondering is

@Shanna517

Posted in: #Bandwidth #Server #WebCrawlers

I would like to operate a website crawler/scraper from my web server (doing so ethically of course).

What I am wondering is whether this uses my website bandwidth up? If so how badly and is it possible to minimise bandwidth usage? I am currently assigned 5000GB per month website bandwidth.

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Shanna517

1 Comments

Sorted by latest first Latest Oldest Best

 

@Sarah324

Anything that requests a resource (a.k.a. web page, image file, stylesheet, etc) from your server uses bandwidth. So crawling other sites from your server will count against your bandwidth limit.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme