The Big Data and Cloud Computing Trends Depend on Open Source

Reuven Cohen has an interesting post up on Forbes' site, which asks, "Free Versus Open: Does Open Source Software Matter in the Cloud Era?" He writes: "I like open source as much as the next guy but, from a value proposition standpoint, just being 'open source' doesn’t sound all that compelling to me.  This has become especially true in the emerging cloud computing landscape where APIs and Big Data have become some of the most valuable currencies." In fact, though, as the transition to the cloud and Big Data continue, open source software is playing an absolutely critical role.

Cohen notes that Big Data has become one of the "most valuable currencies," but isn't the open source Hadoop platform--used to sift insights from extremely large data sets--one of the flagship pieces of software driving the Big Data trend? Hadoop has given rise to promising startup companies such as Hortonworks, focused on training and services surrounding it. 

Meanwhile, on the cloud computing front, open source platforms such as OpenStack, CloudStack and Eucalyptus are presenting important alternatives for companies that want more flexibility than proprietary clioud platforms provide. There is a whole ecosystem of open source tools taking shape around these.

And, as cloud computing proliferates, let's not forget that Google, Yahoo! and many of the most successful Internet companies have built themselves around open tools, and made important contributions to FOSS. Let's take Yahoo, for example. 

In 2009, Yahoo donated Traffic Server scalable caching proxy to the Apache Software Foundation, and the cloud computing team at Yahoo contributed this guest post to us here at OStatic about it. Traffic Server is used in-house at Yahoo to manage its own traffic and it enables session management, authentication, configuration management, load balancing, and routing for entire cloud computing stacks.

Yahoo has always had developers and APIs deeply focused on PHP, Java, JavaScript, AJAX, ColdFusion, Ruby, Python and much more, in addition to many cross-platform applications. The company has also been open about sharing user interface tools, as you can see at the Yahoo! User Interface Library (YUI).  And, Yahoo's site was built from day one to run on FreeBSD technology. Beyond that, Yahoo is recognized for developing its own distribution of Hadoop technology for querying huge data sets and contributed mightily to the advancement of Hadoop.

Neither Big Data nor cloud computing would be the same today without these efforts, where open source software has been critical to progress. These trends are relatively young, too, and open source software will stay critical as they mature.