Gradle: Just One Example of LinkedIn's Open Source Interests

The big news on the business front today is that Microsoft is buying LinkedIn for $26.2 billion in cash. The deal values LinkedIn at $196 per share, which is a 49.5 percent premium over Friday's closing price. Some analysts are wondering if Microsoft got a good deal.

One of the things that no doubt attracted Microsoft to LinkedIn is that LinkedIn not only leverages many open source projects at huge scale, but it releases interesting projects as open source as well. As just one example, developers at LinkedIn internally created a project called Gradle, comprised of several workflow tools that simplify connecting multiple Hadoop jobs within the context of an application. It's making a difference for several companies, and here is more about this interesting tool.

From Google to Netflix to Facebook, many large tech companies are good citizens when it comes to open sourcing tools that can be widely used by others, and LinkedIn has been joining the crowd.

The LinkedIn Gradle Plugin for Apache Hadoop ("Hadoop Plugin") includes the LinkedIn Gradle DSL for Apache Hadoop ("Hadoop DSL"). You can get the Hadoop Plugin on Github.

LinkedIn has already adopted Gradle as its primary build system. "With Gradle, developers can easily extend the build system by defining their own plugins," the company claims. "We developed the Hadoop Plugin to help our Hadoop application developers more effectively build, test and deploy Hadoop applications. The Plugin includes the Hadoop DSL, a domain-specific language for specifying jobs and workflows for Hadoop workflow managers like Azkaban and Apache Oozie."

A post from Alex Bain adds:

"In particular, the Hadoop Plugin includes tasks that will help you more easily work with a number of Hadoop application frameworks. Since no one tool is perfect for every kind of job, Hadoop jobs at LinkedIn are written using a number of different application frameworks. The Hadoop Plugin enables developers to organize their Hadoop projects in a consistent fashion regardless of the particular tool they choose for the job."

"Long before the Hadoop Plugin, Hadoop developers at LinkedIn had realized that writing individual Hadoop jobs was only part of the challenge in using Hadoop effectively. Most data-driven features that appear on LinkedIn are actually generated by processing pipelines that may consist of dozens of individual Hadoop jobs chained together into workflows managed in Azkaban or Oozie. Understanding the relationships between jobs in a workflow and managing the workflow specification files became a challenge in itself."

 You can find out much more about Gradle here.

Microsoft's acquisition of LinkedIn will be interesting to watch. The Redmond giant has been upping its game in the open source arena, and will inherit a number of open projects as it brings LinkedIn into the fold. We'll stay tuned.