Exploring SmartOS
Continuing along in the vein of exploring all the options in datacenter virtualization, my journey has led me, unavoidably some might say, to Joyent’s SmartOS. SmartOS is a decedent of Solaris, one of the first Unix systems I learned on over a decade ago. Being based on Solaris, and on account of a number of other features, SmartOS is definitely a horse of a different color.
SmartOS is based on Illumos, which is a fork of the short lived OpenSolaris distribution from Sun. OpenSolaris was based on Solaris, which was based on SunOS, which was a combination of BSD and SVR4. In other words, this is real Unix, modernized, battle tested, and focused. On my drive to work this morning I was thinking about all the different platforms that a Linux kernel needs to be aware of. Of course, not every device runs the exact same kernel, but the kernel still needs to be able to run on big iron servers, workstations, desktops, laptops, tablets, cell phones, wifi routers, and probably toasters too. Linux has a big job, and a lot of responsibilities. SmartOS, by comparison, has only one.
SmartOS was designed for one thing, to run the datacenter, a fact evident as soon as you try to “install”. I put install in quotes there because you don’t actually install SmartOS. The entire distribution comes as a live, mostly read-only image. This threw me for a loop when we decided to install on a couple of spare machines today. I booted up the disk, saw the root login prompt, and decided that I had downloaded the wrong image a couple of months ago. So, we took a break while we downloaded another image, and I read through the documentation. (Pro tip: read the documentation)
SmartOS is run entirely from RAM. The entire operating system is loaded into RAM and run without a root partition on a disk. The entire /etc directory is overwritten at boot time, and the only configurations that persist are the bare minimum of the hostname and networking configuration. The entire OS takes up approximately 250MB in RAM. This setup provides some interesting options for datacenter architecture.
Since SmartOS is run entirely from RAM on a read-only image, the real work of the OS is handled in zones, analogous to FreeBSD jails or OpenVZ containers. The real beauty in the zones is how SmartOS handles how they are built. SmartOS inherited some fantastic technology from Solaris, and some brilliant engineers from Sun. One of the foundations of SmartOS, and one of the most interesting technologies to come out of Sun, is the filesystem, ZFS. I loved working with ZFS in FreeBSD, and I’m glad to have the opportunity to explore it more. SmartOS creates a new ZFS filesystem for each zone. Couple the capabilities of ZFS like snapshots, streams, and quotas with zones and you have a highly efficient means of deploying and backing up your applications in production.
SmartOS also ported KVM, which allows provisioning of other operating systems on the SmartOS platform. You can run Ubuntu, FreeBSD, or even Windows, and gain many of the benefits of SmartOS. Each virtual machine is presented with it’s own ZFS volume, which it recognizes as a block device (like a hard drive). The VM then formats the volume as whatever it needs it to be and happily installs.
After installing, er, provisioning our first node, we installed a very nice web-based GUI named FiFo. FiFo seems very well designed, and while it may not be quite as full featured as other GUI cluster management software, what it does do it seems to do well. So far, my experience with SmartOS has been positive. Returning to a pure Unix system feels good, and I have high hopes for SmartOS in our environment. And I haven’t even gotten to Dtrace.
SmartOS is very different, but I think it is worth some attention. After all, it is based on one of the most reliable data center systems in the world, and the only OS I know of to reach over ten years of uptime. I’ve seen some of my systems get over three, and a NetApp that might be reaching four, but a decade is crazy.