Twitter and Tech Titans End Year with Big Open Source Offerings
The end of 2015 marked the open sourcing of some very promising tools from tech giants, including Google, Facebook and Twitter. Recently, I covered Google's decistion to open source a program called TensorFlow. It’s based on the same internal toolset that Google has spent years developing to support its AI software and other predictive and analytics programs. And, we reported on how H2O.ai, formerly known as Oxdata, has announced a new funding round that it is getting to the tune of $20 million. The money will go toward advancing its machine learning toolset.
And, Facebook is open sourcing its machine learning system designed for artificial intelligence (AI) computing at a large scale. It's based on Nvidia hardware. Meanwhile, Twitter has open sourcced Diffy, which is software that developers can employ to ferret out bugs when they’re making updates to certain parts of code. Diffy is now available on GitHub here.
A Twitter blog post explains what Diffy is designed for:
"Today, we’re excited to release Diffy, an open-source tool that automatically catches bugs in Apache Thrift and HTTP-based services. It needs minimal setup and is able to catch bugs without requiring developers to write many tests. Service-oriented architectures like our platform see a large number of services evolve at a very fast pace. As new features are added with each commit, existing code is inevitably modified daily – and the developer may wonder if they might have broken something."
"As the complexity of a system grows, it very quickly becomes impossible to get adequate coverage using hand-written tests, and there’s a need for more advanced automated techniques that require minimal effort from developers. Diffy is one such approach we use."
As for Facebook's newest open source offering, the company's Kevin Lee and Serkan Piantino wrote in a blog post that the open sourced AI hardware more efficient than off-the-shelf options because the servers can be operated within data centers based on Open Compute Project standards.
They noted the following:
"At Facebook, we've made great progress thus far with off-the-shelf infrastructure components and design. We've developed software that can read stories, answer questions about scenes, play games and even learn unspecified tasks through observing some examples. But we realized that truly tackling these problems at scale would require us to design our own systems. Today, we're unveiling our next-generation GPU-based systems for training neural networks, which we've code-named 'Big Sur.”
"Big Sur is our newest Open Rack-compatible hardware designed for AI computing at a large scale. In collaboration with partners, we've built Big Sur to incorporate eight high-performance GPUs of up to 300 watts each, with the flexibility to configure between multiple PCI-e topologies. Leveraging NVIDIA's Tesla Accelerated Computing Platform, Big Sur is twice as fast as our previous generation, which means we can train twice as fast and explore networks twice as large. And distributing training across eight GPUs allows us to scale the size and speed of our networks by another factor of two."
"We want to make it a lot easier for AI researchers to share techniques and technologies. As with all hardware systems that are released into the open, it's our hope that others will be able to work with us to improve it."
You can find out more about Big Sur here.
Google is also making a possibly hugely influential contribution to the field of machine learning with its latest open source tool. It is open sourcing a program called TensorFlow that will be freely available. It’s based on the same internal toolset that Google has spent years developing to support its AI software and other predictive and analytics programs.
You can find out more about TensorFlow at its site, and you might be surprised to learn that it is the engine behind several Google tools you may already use, including Google Photos and the speech recognition found in the Google app.
According to Google, TensorFlow could help speed up processes ranging from drug discovery to processing astronomy-related data sets.