Open Source and the Poor Man's Supercomputer

IBM made a slew of announcements and predictions on open source topics at this week's LinuxWorld conference, but one of the least talked about and written about announcements was this one (Matt Asay did single it out). IBM released its HPC Open Software Stack as open source, aiming at cheaper-to-deploy, simpler supercomputers. The software stack is  designed to make clusters--servers linked together to form a single super-fast system--more productive, less costly in both software and hardware terms,  and easier to manage. Why should the open source community at large pay attention to this?

IBM knows a thing or two about supercomputers. As you can confirm at Top500.org the company's Roadrunner supercomputer is the world's fastest computer, with a peak performance of more than 1 petaflop/s (one quadrillion floating-point operations per second). The company has also worked on supercomputers that can dual-boot Windows and Linux.

Supercomputers are rarely written about by the mainstream technology press and on technology blogs, but they are vitally important to many forward-looking efforts. I've been to supercomputing conferences where people from the pharmaceutical industry have said that entire diseases will be controlled by designer drugs as soon as we reach certain levels of processing power for drug modeling. Likewise, I've heard astronomers say that we will unlock key new information about the universe when we reach certain processing thresholds. We need faster supercomputers for much more than just national defense and nuclear bomb simulations.

While it wasn't an open source story, I thought this story about some researchers at the University of Antwerp making a supercomputer based on four nVidia graphics cards, was a supercomputing story reflecting the open source spirit. The researchers used four nVidia GeForce 9800 GX2 cards--the inexpensive, off-the-shelf cards that folks like gamers favor--to lash up a system that rivals the speed of clusters consisting of hundreds of computers.

Briefly, their system was so fast that it ranked among the top 10 supercomputers in the world. Check out one of the researchers explaining the system in a video clip here. Can you say....geek? He's awesome.

IBM's new open source software stack "can help develop and execute applications as well as manage and monitor a system," according to its announcement. It also includes IBM's Extreme Cluster Administration Toolkit (xCAT)--the very same administration tools that drive chart-topping supercomuter Roadrunner.

It would be good to see more open source contributions to the arena of supercomputing, and more open source-like thinking from smart hardware people. Those University of Antwerp researchers were just hypothesizing that newer, super-fast GPUs (graphics processing units)--found on cheap graphics cards--could work in combination to compete with clusters of far more CPUs (central processing units). They were right, and more power to them. That kind of inventive hardware thinking, in combination with cutting-edge, open source supercomputing software, could lead to some very good things.