Amazon Joins Tech Giants in Open Sourcing a Key Machine Learning Tool
Among technology categories creating sweeping change right now, cloud computing and Big Data analytics dominate the headlines, and open source platforms are making a difference in these categories. However, one of the biggest open source stories of the year surrounds newly contributed projects in the field of artifical intelligence and the closely related field of machine learning.
Some of the biggest tech companies are helping to drive the trend. Google has open sourced a program called TensorFlow. It’s based on the same internal toolset that Google has spent years developing to support its AI software. Meanwhile, Facebook is open sourcing its machine learning system designed for artificial intelligence (AI) computing at a large scale. It's based on Nvidia hardware. And, IBM announced that its proprietary machine learning program known as SystemML is freely available to share and modify through the Apache Software Foundation. Now, Amazon has contributed a project in this space, called DSSTNE.
DSSTNE is available now on GitHub under an Apache 2.0 open-source license.
According to Amazon:
"DSSTNE (pronounced "Destiny") is an open source software library for training and deploying deep neural networks using GPUs. Amazon engineers built DSSTNE to solve deep learning problems at Amazon's scale. DSSTNE is built for production deployment of real-world deep learning applications, emphasizing speed and scale over experimental flexibility.
DSSTNE was built with a number of features for production workloads:
Multi-GPU Scale: Training and prediction both scale out to use multiple GPUs, spreading out computation and storage in a model-parallel fashion for each layer.
Large Layers: Model-parallel scaling enables larger networks than are possible with a single GPU.
Sparse Data: DSSTNE is optimized for fast performance on sparse datasets. Custom GPU kernels perform sparse computation on the GPU, without filling in lots of zeroes.
There is a User Guide for detailed information about the features in DSSTNE. You can also check examples to start planning your own Neural Network Modeling using DSSTNE.
According to a FAQ:
"Deep Scalable Sparse Tensor Network Engine, (DSSTNE), pronounced “Destiny”, is an Amazon developed library for building Deep Learning (DL) machine learning (ML) models. DSSTNE significantly outperforms current open source DL libraries on certain tasks where training data is sparse (where almost all the values are zero).
We are releasing DSSTNE as open source software so that the promise of deep learning can extend beyond speech and language understanding and object recognition to other areas such as search and recommendations. We hope that researchers around the world can collaborate to improve it. But more importantly, we hope that it spurs innovation in many more areas.
DSSTNE has been built from the ground up to support use cases where training data is sparse, while other libraries such as Caffe, TensorFlow, Theano and Torch have a larger feature set and network support. DSSTNE resembles Caffe in spirit, emphasizing performance for production applications. DSSTNE is much faster than any other DL package (2.1x compared to TensorFlow in 1 g2.8xlarge) for problems involving sparse data, which includes recommendations problems and many natural language understanding (NLU) tasks. DSSTNE is also significantly better than other packages at using multiple GPUs in a single server. DSSTNE can automatically distribute each computation across all available GPUs, speeding up all computations, and also enabling larger models than other packages can build without heroic effort. Practically, this means being able to build recommendations systems that can model ten million unique products instead of hundreds of thousands, or NLU tasks with very large vocabularies."