Making Money with Web Scraping – ScrapingFish
As you probably know by now, I’m a fan of founders who see an opportunity to make money by web scraping, and create a business around that product. Today, I’d like to introduce you to a company that was already in business, and saw a bigger opportunity in something they’d built for their own use.
The founders of ScrapingFish, Mateusz Buda and Paweł Kobojek, met when they were studying computer science, and latched onto some of the concepts they were learning in the networking courses. They began working together in contracting, and they still run a consulting practice as well as their product business. Let’s take a closer look.
What does ScrapingFish do?
ScrapingFish provides proxies for people and companies who scrape data as part of their business. They’re not data center IPs, though. They’re rotating mobile proxies, along with a browser cluster that uses those proxies. Their target market is a technical user – a programmer who needs data but doesn’t want to run their own proxy cluster.
Buda and Kobojek knew there was a better way to ensure secure access to data from the web. Data center IPs are too easily blocked by sites that house valuable data. Residential IPs are somewhat better, but mobile proxies are the hands-down best. They’re best because they’re constantly changing. Mobile proxies are ephemeral, connecting to a different BTS (Base Transceiver Station) even when you’re in a static location. They’re also highly unlikely to be blocked. By blacklisting a mobile IP, a site would ban legitimate users in the process.
There’s also an ethical issue at play. Proxy providers frequently source IPs using unethical means. From day one, Buda and Kobojek committed that ScrapingFish would ethically source every IP they use.
Where ScrapingFish began
Like the other companies featured here, ScrapingFish was rooted in a client’s need for data. Buda and Kobojek were operating a software company – NARF.ai – and they had clients that needed to gather large amounts of data from the web. They built the infrastructure for ScrapingFish initially for internal use, to allow them to better serve those clients. But that was only the beginning. They quickly saw the opportunity that existed in scaling the infrastructure they built, making it available publicly.
Here’s the brilliant part of that scaling model: because using the API requires some level of programming skill, their product can be used by hundreds – even thousands – of users without significant direct involvement on their part. They can scale the business without having to scale their time.
How is ScrapingFish doing?
In short, they’re doing very well, with growth month over month, and with more nodes and modems being added all the time. The next stage in their growth is expanding to other countries where there is demand for their product. The challenge in that case will be the geographical limitation of mobile proxies – the fact that the IP has to be physically located in a specific place – but Buda and Kobojek have a plan to make it happen.
Interestingly, even as ScrapingFish grows and becomes more successful, the founders don’t plan to leave consulting behind. As we saw, ScrapingFish can scale without a significant drain on time. Moreover, ScrapingFish acts as a business development funnel, attracting companies that are prospects for their consulting business.
Want to try it yourself?
Mobile proxies are clearly the way to go if you want to make money web scraping. If ScrapingFish has inspired you to take a closer look, Buda and Kobojek have you covered. On their website, they’ve written a detailed rundown showing you how to build your own mobile proxy using a Raspberry Pi and a SIM card. Check it out for all the hardware setup and code you need to try this out on a small scale.
Why, you might ask? Rotating through proxies is a hassle. Imagine setting up your workflow so that a new IP is generated when you get blocked. How much more could you get done? Using their setup, you can even create your own cluster with multiple modems so that more concurrent requests can be made from different IPs.
It’s on a smaller scale, but it’s a great project. And you never know, you might discover that working with the larger infrastructure ScrapingFish brings to bear is the next step for your web scraping. I talked to Mateusz Buda and Paweł Kobojek about the DIY project, and you can find that video here. I also talked to them about ScrapingFish – where it came from and where it’s going – and you can find that conversation here. Check them both out, and don’t forget to like and subscribe when you’re there, for more great ideas to make money by web scraping.