Netflix Keeps Open Sourcing its "Simian Army" of Useful Cloud Tools
While cloud computing platforms make headlines every day now, including leading open source platforms such as OpenStack, it's still true that cloud computing is a young science. There is a premium on reliable, mature tools for the cloud, and a real need for tools that can usher in better security. Also, it's true that Amazon Web Services (AWS) is still the 800-pound gorilla in the cloud.
Los Gatos, Calif.-based Netflix is one of the many companies that has been making extensive use of cloud services and tools for years, and we've reported on Netflix open sourcing a series of interesting "Monkey" cloud tools that it has deployed as satellite utilities orbiting its central cloud platform. It has released a new one of these, and all of them are worth taking note of.
Netflix has been using its own tool for staying secure as engineers with various accounts change configurations for aspects of the Netflix platform atop Amazon Web Services. The Security Monkey tool has worked well for the company, and the company has blogged about its open source availability here. According to the post:
"We envisioned and built the first version of Security Monkey in 2011. At that time, we used a few different AWS accounts and delivered the service from a single AWS region. We now use several dozen AWS accounts and leverage multiple AWS regions to deliver the Netflix service. Over its lifetime, Security Monkey has evolved (no pun intended) to meet our changing and growing requirements."
"Security Monkey is relatively straightforward from an operational perspective. Installation and AWS account setup is covered in the installation document, and Security Monkey does not rely on other Netflix OSS components to operate."
"At Netflix, when we analyzed our Amazon Web Services (AWS) usage, we found a lot of unused resources and we needed a solution to rectify this problem. Diligent engineers can manualy delete unused resources via Asgard but we needed a way to automatically detect and clean them up. Our solution was Janitor Monkey."
"Janitor Monkey is a service which runs in the Amazon Web Services (AWS) cloud looking for unused resources to clean up. Similar to Chaos Monkey, the design of Janitor Monkey is flexible enough to allow extending it to work with other cloud providers and cloud resources. The service is configured to run, by default, on non-holiday weekdays at 11 AM. The schedule can be easily re-configured to fit your business' need. Janitor Monkey determines whether a resource should be a cleanup candidate by applying a set of rules on it."
As for Chaos Monkey, what the program does is randomly kill instances within Netflix's architecture, working on the assumption that constant failures will help build robust defenses against catastrophic failure. It's an interesting fault-tolerance tool.
You can peruse Netflix's overall open source software resource center on GitHub. The company is steadily releasing proven tools that can be quite useful for cloud administrators. Netflix has also said that it has more tools from its "Simian Army" due to be open sourced soon.