Recent Bug Report Details Data Loss in ext4; Ts'o Explains Cause and Workarounds

By way of Heise Online comes more detailed information about what's causing some to lose data in systems using the new ext4 file system. The ext4 file system, which is slated to eventually replace ext3 as the default file system of choice in many Linux distributions, can experience significant data loss after system crashes or unclean shutdowns.

Ted Ts'o, Linux Foundation's CTO and maintainer of the ext4 file system, says that the loss is due to how ext4 writes to disk. He says ext4, and newer file systems such as XFS and Btrfs, use delayed allocation. This means that writing data to disk can take up to a minute (compared to ext3's write time of about five seconds). The longer write time improves system performance and organization of data on the hard drive. Ts'o says that the ext4 team is working out a solution, but it's an issue that would benefit from being addressed from all angles.

The bug report was filed in Ubuntu's bug tracker for the upcoming Jaunty Jackalope release. Jaunty offers ext4 as an optional file system choice (the Ubuntu project expects that October's Karmic Koala will feature ext4 as the default). Shortly after the desktop environment loaded, a system crash occurred, destroying a significant amount of data (including desktop configuration files).

Ts'o says this was due, in part, to the delayed allocation function not having enough time to write the data to the disk. While patches can help, and at least a partial solution is in the works (and should be integrated after the next merge window), these fixes are not in the mainline kernel yet, nor are they in a kernel supplied with the Jaunty alpha version.

Being that many common distributions offer ext4 only as an option, and a number of those distributions are only doing so through testing releases, this is an issue that is largely confined to non-production machines and will be sufficiently in hand before the new file system is widely deployed. Ts'o says one way to secure data on ext4 machines is to simply make sure the system is stable. Of course, this is tricky when testing an alpha or beta release, but in these settings crashes -- perhaps accompanied by data loss -- are to be expected.

Ts'o says the fixes currently in the works should preserve the performance enhancements and data organization functions of ext4 while minimizing the chance of data loss. He says ideally, application and desktop libraries should be reorganized, utilizing small database structures in lieu of many small files in private home directories.