Open Source, Big Data, and the Governance Challenge

Recently, John Schroeder, executive chairman and founder of MapR Technologies, Inc., one of the top players in the Big Data arena, was kind enough to give us his predictions on open source anb Big Data topics for 2017. He noted the following: "In 2017, the governance vs. data value tug of war will be front and center."

Indeed, how organizations handle everything from their customer data to their transactions is in flux, and now more interesting predictions on the topic of data and governance are coming in.

Schroeder added that regulated and non-regulated data governance issues are in play: 

"Leading organizations will manage their data between regulated and non-regulated use cases. Regulated use cases data require governance; data quality and lineage so a regulatory body can report and track data through all transformations to originating source. This is mandatory and necessary but limiting for non-regulatory use cases like customer 360 or offer serving where higher cardinality, real-time and a mix of structured and unstructured yields more effective results."

 The issue spills over to how organizations govern open source code as well. According to a recent post from The VAR Guy:

"In elder days, most open source code was written by volunteers. Today, the vast majority of code contributions to projects like Linux and OpenStack come from programmers paid by companies like Red Hat and Intel. There's nothing wrong with this; the fact that companies are investing so much money in open source development is a good thing. But this change does reflect a much higher degree of corporate control over open source code. That leads to tensions that the open source community must contend with."

AtScale's 2016 Big Data Maturity Survey, based on 2,550+ responses from big data professionals at 1,400 companies across 77 countries, also reveals governance issues taking shape. The survey found that accessibility, security and governance have become the fastest growing areas of concerns year-over-year, with worries related to governance growing the most at 21%.

In 2017, organizations will shift from the “build it and they will come” data lake approach to a business-driven data approach, Schroeder wrote in his set of predictions. That's just one of the ways that data and open source governance issue will rise to the top as this year progresses.