Google Delivers Octane, An Update to Its V8 JavaScript Benchmark Suite

All the way back in 2010, we covered Google's release of its WebM royalty-free video codec, and the release of its underlying V8 technology under an open source license. Along with V8, Google developed a suite of industry standard V8 benchmarks, which are widely used to test JavaScript performance. Now, the company has delivered Octane v1, an enhanced suite of tests.

As explained in a new post from Google:

"The web is evolving and so should the JavaScript benchmarks that measure its performance. Today, we are releasing Octane, a JavaScript benchmark suite that aims to measure a browser’s performance when running the complex and demanding web applications that users interact with daily...Most of the existing JavaScript benchmarks run artificial tests that were created on an ad-hoc basis to stress a specific JavaScript feature. Octane breaks with this tradition and extends the former V8 Benchmark Suite with 5 new benchmarks created from full, unaltered [1], well-known web applications and libraries. A high score in the new benchmarks directly translates to better and smoother performance in similar web applications."

The new tests are as follows:

Here is an overview of the new tests, from Google:

• Box2DWeb runs a JavaScript port of a popular 2D physics engine that is behind many well-known simulations and web games.
• Mandreel puts a JavaScript port of the 3D Bullet Engine to the test with a twist: The original C++ source code for the engine is translated to JavaScript by Onan Games’ Mandreel compiler, which is also used in countless web-based games. 
• Pdf.js is based on Mozilla’s PDF reader and shows how Javascript applications can replace complex native browser plug-ins. It measures how fast the browser decodes a sample PDF document. 
• GB Emulator is derived from an open source emulator of a famous game console running a 3D demo. 
• CodeLoad measures how quickly a JavaScript engine can bootstrap commonly used JavaScript libraries and start executing code in them. The source for this test is derived from open source libraries (Closure, jQuery).

 It does look like Google has constructed these tests to mirror performance in actual web applications, which could be an improvement on the primarily abstract tests that are often used to measure JavaScript performance. And, increasingly, JavaScript speed is the metric that users want to focus on when evaluating performance in browsers.

It's also worth noting that you can easily run Octane on a smartphone or tablet, and the interface is designed to accommodate these devices.