15 Oct

News Hub facts you wanted to know about

The system works on HCE-DC – multipurpose high productivity, scalable and extensible engine of internet data mining.

It consists of several HCE project’s sub-products and technologies:

  1. HCE-node – network transport cluster application
  2. Distributed Crawler (DC) service
  3. Distributed Tasks Manager (DTM) service
  4. Web administration management console
  5. Tools and libraries for crawling and scraping algorithms with REST API and bindings for a Python and PHP development environments.

It provides flexible configuration, automated deployment, and easy integration with 3rd party data mining and analysis projects.

Share this