Scraper Requirements

Here is the basic requirements for Scraper plugin :

1 GB RAM
1 CPU Core
25 GB SSD Storage
1 TB Transfer
40 Gbps Network In
1000 Mbps Network Out

We use Digital Ocean’s base server option for testing. The table above captured from Linode.com’s base server.

If you use dedicated virtual cloud server for Scraper plugin, that will be the perfect choice scraping.

The main problem with scraping process, when you use a shared hosting, it will use shared resources. RAM, CPU or network can be used from other customers time to time. That can affect plugin’s performance.

Also these companies provides dynamic IP addresses that can be replaced with an another. And this availability can be useful in any ban or block cases.

So we simply suggest to you use virtual dedicated server. Even small ones will provide very good performances. We currently use Linode and DigitalOcean’s base plans for tests.

https://www.linode.com/pricing

https://www.digitalocean.com/pricing/