Rapture3D for Virtual Reality

We've had quite a few enquiries recently about our using new Rapture3D game audio engine, and thought we should let more folk know where we are with this.

In summary: it works, it's fast, it's portable, and it has some cool new features which are particularly relevant to VR.

The Library

The new engine is a software library designed to be integrated into games or VR software by developers. It uses its own simple native C interface rather than the OpenAL interface used in previous versions. It is "buffer-based", so you have direct control of each block of audio as it passes through the software. This also makes the engine easier to integrate into more complex audio pipelines.

The code is fast, with SSE and NEON optimisations, and is tested on Win32, Win64, OS X, Linux, iOS and Android so far. There's middleware integration into Unity using a C# wrapper, and hopefully more soon. The engine and its data are now contained in a single software library for much simpler deployment.

Rapture3D in Unity

Keeping the Best

The high fidelity signal path, directional reverbs and high quality sample rate conversion have been carried over from previous versions of Rapture3D (with some improvements and optimisations) and we've now extended support on our HOA busses up to fifth order as standard, for even better spatial imaging. The available decoders ("output renderers") include stereo, 5.1, 7.1 and a number of HRTF-based binaural headphone decoders, including one using our Amber HRTF.

This will be obvious to those who know our stuff, but it's probably worth mentioning some specific features:

  • Performance! Because of the radically different architecture used, performance scales in a different way to other HRTF engines. You can spatialise large numbers of sounds, even on mobile, and the CPU load can be fine-tuned by controlling the HOA order.
  • Quality! You'll really struggle to find any discontinuities on block boundaries here, and the HRTFs used are very smooth. There are no special settings needed to handle artefacts. Our Amber HRTF is rather good, and there are alternatives if this isn't ideal for your head shape.
  • Consistency! The HRTF engine is just a part of the Rapture3D ecosystem. If you want to render to stereo, 5.1 or suchlike, this is just a matter of selecting a different decoder.

Detailed 3D Audio Scenes - with Head Tracking

There are a number of new features particularly well-suited to VR, particularly around "360 Video". In particular, audio prepared using ambisonics can be used directly as multichannel assets ("beds"), at up to third order. This means that you can create a highly detailed TOA 3D audio mix in the studio using our TOA pro audio products and then feed it directly into Rapture3D for rendering as a complete VR audio scene, or as part of a larger one.

This makes it possible to add any amount of complex sound material into a single TOA mix, add multichannel recordings, and work on subtle details of the overall mix until it's absolutely right (in the much the same way we're used to for stereo or 5.1, but in full 3D). This gives a lot more control than is possible with pure "object-based" approaches, particularly where reverb or fine sculpting of the overall mix is concerned. Of course, if you want to use individual sound sources too you can, and Rapture3D's design scales well to very large numbers of ordinary sources, with dynamic reverb, filtering and distance models.

And because of the way ambisonics works, Rapture3D can rotate a complete TOA 3D audio mix easily to compensate for the direction in which the user is actually looking (and listening) without loss of quality. With the low-latency head trackers in modern VR headsets it's easy to not even notice this happening; the audio scene stays stable when the head turns. And as a final "trump card", Rapture3D can actually translate the 3D mix, so you can walk out of one and into another!

If you're writing a VR game or 360 video player, please feel encouraged to get in touch to find out more...