Why Webscale Companies Need open Source

Those who do not remember the past are condemned to repeat it. George Santayana wasn't talking about open source when he said that, of course, but it applies to the lessons of proprietary software just as well. Those who haven't learned why open source is important are doomed to rely on proprietary solutions. Luckily, the companies that haven't learned are getting further and further between.

But some folks are still struggling with the lessons of the bad old days of proprietary software ruling the roost. Over on OStatic sister site GigaOM, Derrick Harris suggests that companies are "making more work for themselves than is necessary" by choosing to support their own open source projects:

Isn't it plausible that a proprietary vendor –- Oracle, let's say –- could launch a webscale database or analytics solution that would do the trick for a company like Facebook? If there's one thing Larry Ellison knows better than relational databases, it's how to make a buck. Hypothetically speaking, Oracle could offer database and data-analysis solutions that could save a company like Facebook from having to act like a software company itself. It certainly hasn't hesitated to buy its way into alternative markets in the past.

Let's start with the obvious. Facebook is a software company. So is Twitter. So are many of the other companies that are participating in open source infrastructure development. They're not in the business of selling software, but the companies couldn't outsource their development and still retain the control and speed they need to do business in a space that requires constant evolution. Developing software is one of their core competencies.

Oracle has the competence to develop the software that Facebook and the others need, but does it have the interest? How many companies, aside from Facebook, would be willing to buy the same software? Oracle's database offerings are aimed at a very large market with a fairly common set of problems. Facebook's problem set is different than the average corporate customer or organization that Oracle caters to. It doesn't seem likely that it would cost Facebook less to go to a proprietary vendor and ask for a custom solution. Maybe the solutions are "emerging" from proprietary vendors, but they haven't emerged yet and it would take longer to work with a vendor like Oracle than just doing it yourself.

And Harris overlooks the benefits of open source entirely. Companies invest in open source not only to save on licensing fees, but to control their own destiny. What happens to Facebook when it can't get at the code to fix a problem that's causing massive downtime? Even if it costs more for Facebook to develop its own software than to work with a proprietary vendor (which is not by any means a given), the extra costs may be worth the additional control.

Facebook, et. al., would not be possible today if it weren't for open source software. Commodity hardware and open source software have provided the fertile breeding ground for Web-scale sites like Facebook, Twitter, Reddit, and others. Of course they're going to turn to open source for the next generation of software. Had the initial stack of software they rely on been proprietary, their existence wouldn't have been possible. But these companies have enjoyed the control and flexibility that open source enables and they are wisely choosing to invest their profits into more of the same.

Web companies should absolutely, and fully, commit themselves to rolling their own code or hitching their wagon to existing open source solutions. The alternative is to cede an unhealthy amount of control over their infrastructure to outside parties.