Scrapinghub Bewertungen

Vorteile

It really saves my time as a web scraping specialists. I always need rotated proxies for my client needs. I keep creating a new project and it is a lot of hassle of I have to buy proxies, renew proxies, and create proxy rotation on my own. I am glad I don't even need to think about it. It just works!

Nachteile

Well, it's just doesn't have enough advanced documentation on Scrapy Splash combined with Crawlera. Their development team in Github is very responsive though.

Vorteile

We're only using Crawlera for our data scraping needs and it has worked seamlessly since we started using it in 2016. When we had to upgrade to a higher limit, the transition was fast, and no dev time was wasted updating our existing codebase.

Nachteile

Pricing can be a bit tricky especially if your need is right in between the plans.

Vorteile

The software delivers the exactly promised result. It has a friendly interface and an efficient support.

Nachteile

You need a roadmap of new features and the pricing of third-party services.

Vorteile

- Original and flexible technology
- No vendor-lock
- Easy to use for professionals
- Pretty convenient API system for integration with third-party solutions

Nachteile

- Not so easy to use by non-professional IT person that still wants to use data scraping
- Lack of ability to create some kind of simple and clear user interface for such a persons
- No simple solution for distributed/high-volume crawling
- Lack of monitoring and alerting, non convenient logging system
- Overpriced Crawlera

Vorteile

We are very happy with Crawlera. It works like a charm with our Scrapy projects, just adding a few lines of code we completely forgot about proxies that ended up failing anyway.

Nachteile

Some requests end up being very slow, but good news is most of them end up going through.

Vorteile

- Works perfectly as long as you do not need Portia
- Fast
- Many available features in one place
- It fulfills my scraping needs well
- EXCELLENT technical support

Nachteile

- Portia (beta) is almost unusable. I got a couple very basic programs out of it before I taught myself how to use Scrapy. Still, if you are looking for a value and have time to deal with the lag/hiccups in Portia, it is by far the best value for a simple point and click scrapying platform that I found. You can make it work, you will just want to put your head through the screen before you are done...

Vorteile

A full integrated platform for a framework well done for it purpose. Easy to use, based on others frameworks structure, so if you are used to web development in python then it's a piece of cake to create spiders.
The integration (scrapy + scrapinghub) its really good, from a simple deployment through a library or a docker makes it suitable for any need.
Good support and constant improvement on the platform.
A lot of plugins and, open to any feature needed.

Nachteile

So far there is nothing I dislike about it.

Vorteile

Scrapy is amazing, python api is easy to use, job dashboard is easy to understand, good documentation and code.

Nachteile

Dynamic pages sometimes hard to scrape, nothing wrong with lua (for splash) as a language, but can be minor barrier to entry for python/javascript users.

Vorteile

I have been using Scrapy and Scrapinghub's services since 2013 and I'm so far very satisfied with their services. Crawlera, their proxy service, works very well! I don't have to setup a proxy farm anymore or configure my scrapers to point to thousands of proxy services as they do all the grunt work for you (it's all automated).

Nachteile

Minor hiccups from time to time on the dashboard. It only affects me when I want to see historical stats but this problem doesn't affect functionality.

Vorteile

This software allows me to access 97% of the web resources I need. Other proxy servers I have used gave me 75% maximum.

Nachteile

That its a time based subscription. I would prefer being able to buy a certain number of requests and renew when those run out.

Vorteile

The best out there for any kind of scraping need. They new AI based scraping api is a game changer

Nachteile

Nothing bad about it. Beats the competition out of the water

Vorteile

Scrapinghub is so easy to use. With few setups you're ready to build your first spider. The integration with github and other addons (such as crawlera) makes things even easier to manage code deploy and proxy network management. Support for Python 3 is also a great improvement

Nachteile

I'm still in early days with scrapinghub and at the moment did not see any big issues to take into account. One things that comes in my mind to be improved is the documentation.

Vorteile

it helps a lot to avoid the self-hosted spiders, this is the primary reason I like it.

and I know it is developing to make it more ease to use, like github integration, however I have set up the CI tools to publish the spider automatically for every code change, so I don't use it.

Nachteile

however it is always a pain that it is more difficult to bypass the bot detection, especially for big website, like amazon, and the other websites protected by Distil. I know this is not the fault of scrapinghub, but it is a big pain. and I know you have a service Crawlera for this purpose, but it is quite expensive.

Vorteile

I use the crawlera proxy system from Scarpinghub. It is very straightforward to use and results are great.

Nachteile

No cons until now. Once in a while a server/IP gets banned but they have a big pool of IPs to serve you.

Vorteile

I use the Python scrapy module to write crawlers to monitor competitors prices. The integration with Github makes it real easy to deploy code to the hub. The dashboards are very useful to monitor progress and schedule jobs. Really happy with this offering.

Nachteile

In the beginning it was somewhat cumbersome to automate data extraction. Once I started using Python scrapinghub module life became easy. So also a pro!

Vorteile

Open source, well documented and very efficient.

Nachteile

Splash needs to be migrated to another tool IMO. I don't find LUA easy to work with as a developer; not because of the language but debugging and maintaining code is really hard. If you use scrapy-splash there is a black box that cannot interact with. I think going mainstream with javascript or python whilst unifying browser scrapping and scrapy itself would be an amazing decision.
So, overall I would love to see a merge between scrapy and a browser scraper.

Vorteile

Your reliability an trust about proxy ip list.

Nachteile

You payment method just accept international credit cards. I used it from Brazil.

Vorteile

As a business once you hit a certain size it makes sense outsourcing webscraping. Scrapinghub has the required specialism to do a good job.

Nachteile

Still needs involved work from the client side to ensure data quality and consistency.

Vorteile

Very easy to use. Great dashboards for monitoring jobs. Very competitive pricing. Friendly customer service.

Nachteile

There's very little to complain about with Scrapinghub - it's genuinely a really great product!

Vorteile

Crawlera makes it really easy for me to process thousands of requests without worrying about rate-limits. I love that I don't have to worry about retry policies or IP blacklisting because Crawlera takes care of that for me! Plus, by charging by requests instead of bandwidth, it's much easier for me to estimate my expenses based on my needs.

Nachteile

I don't like that the usage numbers are delayed by so many hours.

The pricing is also a little high. I wish there was some way to bring that down or get more value into the plans.

Vorteile

The effectiveness - it simply works better and is more consistent than other proxies.

Nachteile

The monthly limit on requests and how there is no intermediary plan between C10 and C50.

Vorteile

Simplicity, no-bullsh*t services, modular services & addons, very low service downtime. I can keep going and still won't do these guys justice

Nachteile

Payment platform wasn't a walk in the park. Guides and knowledge base are not outdated but can use a 2019 refresh

Vorteile

It was very easy to start creating crawlers and launching them on scrapinghub. Plug and play. Ready to start launching a fleet of crawlers to move my project to the next level.

Nachteile

The price was a bit hefty for running multiple crawlers.

Vorteile

· Scraping situation is easy to see visually
· Can be finely implemented such as time to activate scraping
· Ease of deployment

Nachteile

· I try to deploy from github, but I can select only my own repository (Can not deploy from organization's repository?

Vorteile

we have used scrapinghub for generating proxy ip's to mine company web data and career page information. has been very useful in assisting us to stand up our own skill vertical search engine.

Nachteile

needs a lot of customization if the scripts run into issues. overall a good product for its use cases