In Acquiring Conductant, Docker Eyes a New Type of Operations Stack

With container technology much in the news, Docker is grabbing headlines as it announced that it has acquired Conductant, a startup company that focuses on orchestration. Bil Farner and David Chung founded Conductant, and Farner is known for his time at Twitter, where he put together the early building blocks of what eventually became the Apache Aurora project.

According to Docker officials: "One very exciting possibility is to integrate Aurora with Docker Swarm to form a powerful large-scale web operations stack."

According to a Docker post:

"Aurora is a popular extension of the Apache Mesos clustering system optimized for extremely large-scale production environments. It is widely recognized as the most scalable and operationally-robust component of the Mesos stack. Aurora is the perfect example of a powerful methodology which we call operations-driven development (ODD). The team at Conductant ran and operated some of the largest cloud environments in the world at Google, Twitter and Zynga. In the process of managing operations for these companies, they had to develop tools that no one else could build for them. This resonated with us because Docker itself is the fruit of our team’s experience operating large-scale cloud platforms. Bill Farner’s team at Twitter built Aurora to meet its own requirement: automate its massive server farms so that they could manage them with just a handful of experienced operations engineers. Very few tools available today have been battle-tested at such operational scale. There are many commercial distributions of Mesos, but none of them incorporate Aurora. We believe that is a wasted opportunity. We plan on incorporating the best ideas from Aurora into Docker Swarm, and are exploring integrating Aurora as an optional component of the official Docker stack."

 In some ways, as the folks at Docker evoke the concept of a "web operations stack" and evoke the names of Mesos and Aurora, what they are focused on sounds similar to what Mesosphere is doing. As we covered here, Mesosphere offers a “data center operating system” (DCOS) built on the open source Apache Mesos project.

"One very exciting possibility is to integrate Aurora with Docker Swarm to form a powerful large-scale web operations stack," writes the Docker team. "Swarm’s goal is to provide robust, standardized primitives for deploying any distributed application, on any infrastructure, at any scale. This makes it an ideal standard base layer to scale all kinds of applications, from databases to stateless web workers, scientific computing or big data pipelines. In contrast, Aurora is optimized for the specific needs of large-scale consumer apps reaching hundreds of millions of users, and developed with a bleeding edge architecture and methodology. We believe that in many use cases, a stack combining Docker Swarm and Aurora could democratize the battle-tested Twitter operations model. Of course, not every application is a good fit for Aurora, and it will remain completely optional for Docker users – and vice-versa. By making two of the most popular open-source infrastructure projects interoperate better, we believe both communities will benefit."