OStatic

Skynet is an open source Ruby implementation of Google’s Map/Reduce framework, created at Geni.com. With Skynet, one can easily convert a time-consuming serial task, such as a computationally expensive Rails migration, into a distributed program running on many computers.

Skynet is an adaptive, self-upgrading, fault-tolerant, and fully distributed system with no single point of failure. It uses a β€œpeer recovery” system where workers watch out for each other. If a worker dies or fails for any reason, another worker will notice and pick up that task. Skynet also has no special β€˜master’ servers, only workers which can act as a master for any task at any time. Even these master tasks can fail and will be picked up by other workers.