EIT Digital to Launch Hadoop-Based Software Framework, and a Startup

by Ostatic Staff - Jan. 20, 2017

While not everyone in the U.S. is familiar with it, EIT Digital is a leading European open innovation organization, and it has just launched a new innovation program called HopsWorks to work on a next-generation Hadoop open-source software framework for distributed storage and processing of very large data sets.

The idea is to leverage Hadoop's Big Data strengths in a new type of software framework, and a whole new startup comany will be created to take it to market.

Dr. Jim Dowling, Senior Researcher at the Swedish Institute of Computer Science (SICS Swedish ICT) and the leader of EIT Digital HopsWorks Innovation Activity, said: "Hadoop is an open-source software framework for storing data and running applications on clusters of commodity hardware. Our product, dubbed 'Hops', will provide the first truly multi-tenant, elastic Hadoop distribution service with unified batch and streaming."

"Unlike existing offerings," he added "Hops will allow organizations to host multiple sensitive datasets on the same cluster, providing dynamic role-based access control for both Hadoop Distributed File Systems - HDFS and Kafka distributed streaming platforms. The platform is unified in an intuitive user interface and provides first-class support for Spark-Streaming, Flink distributed big data analytics, and Kafka."

 EIT's announcement added:

"Hadoop provides massive storage for all kinds of data, enormous processing power and the ability to handle a vast number of concurrent tasks or jobs. It also enables the storage of files that are bigger than what can be stored on one particular node or server."

The Hadoop ecosystem consists of computer clusters built from commodity hardware. All modules in Hadoop are designed with a fundamental assumption that hardware failures are a common occurrence and should be automatically handled by the system.

HopsWorks will launch Hops as a market-ready product and will distribute the framework via new client facing startup focusing on the Internet of Things (IoT) and telecoms markets as well as sensitive Big Data owners. The new startup is expected to gain critical mass and exposure through project partner Ericsson, a global leader in delivering ICT solutions, who will act as the reference user of the framework.

The project will also transfer skills in Big Data and streaming analytics to industry, train skilled workers, and will link with EIT Digital's Data Science Masterschool Education Program and a free course on the edX online learning and MOOC service.

 "We expect to be able to bootstrap the Hops ecosystem, and give the startup the escape velocity required to raise the seed capital needed for building a successful business. We estimate our startup to have substantial revenue growth in 2017 and beyond", Dowling added.