Measuring FAIL: A Scorecard for Evaluating Open Source Projects
Ever wonder whether a given FLOSS project is going to succeed or flounder? Need a little help reading the tea leaves? Now you can follow along at home with a handy scorecard that looks at everything from source control to project communication.
A little history. I was researching a project yesterday that required potential users and contributors to pull source from a Git repository, rather than simply downloading a zip file or tarball with the source and other materials that users would need. I mentioned this on Twitter, and Karsten Wade of the Fedora project pointed out the scorecard on the newly launched guide to helping people learn to interact with and build community, Open Source Way.
The Open Source Way has a ton of material already on how to structure communities, what an organization should be looking at to judge health, and references and books about developing open source software or cultivating communities. It's a wiki, rather than a static reference, so anyone can join in and help build out the guide.
By far the most amusing, yet also fairly accurate, is the How to tell if a FLOSS project is doomed to FAIL page, with a scoring system to tell if a project is going to fail or at least not succeed as well as it could.
Originally developed by Tom 'spot' Callaway, all of the metrics are measured in units of FAIL. Does a project have documentation on how to compile from source? If not, it gets 20 points of FAIL. Does the project require something other than GNU Make to build? That's 10 points of FAIL. Was the project proprietary before becoming open source? That could be up to 50 points of fail, depending on how long the project was proprietary before it make its way into the open source world.
Any score above 30 points is a sign of trouble. Any score above 65 means sure trouble, expressed as "kittens die when your code is downloaded." It's humorous, but also a pretty good set of things to avoid. If you see any of these signs in your project, think strongly about fixing them immediately. And most of them are easy fixes. While you may not be able to do anything about the history of a project, there's no sane reason why a project should be without publicly available source control, or why a project would fail to do versioned releases in a format that's easy to consume.
Want to contribute to the Open Source Way? The site, following its own advice, has some guidelines on how to join in and communicate with the rest of the community.
Joe 'Zonker' Brockmeier is a freelance writer and editor with more than 10 years covering IT. Formerly the openSUSE Community Manager for Novell, Brockmeier has written for Linux Magazine, Sys Admin, Linux Pro Magazine, IBM developerWorks, Linux.com, CIO.com, Linux Weekly News, ZDNet, and many other publications. You can reach Zonker at jzb@zonker.net and follow him on Twitter.