Towards 6DOF: 3D Audio for virtual, augmented and mixed realities

The production of audio for virtual, augmented and mixed reality (‘XR’) is on the verge of rapid growth - disrupting traditional methods of audio production and necessitating the development of new skills, new working methods and new technologies to accommodate these changes. Worldwide adoption of XR headsets and applications has significant growth forecast for the coming decade. Consumer spending is growing, with Sony’s PSVR headset leading the charge, and increased spending can be seen in entertainment, retail, manufacturing, logistics and many other industries. These applications require adaptive, procedural audio that can interact with inputs from users and their environments. XR applications require audio that give semblance to ‘real-world’ sound and that provide spatial information that will guide, inform and entertain the user. This chapter discusses the recent history of procedural and spatial audio and how they combine to produce convincing and entertaining 'Six-Degrees of Freedom' (6DOF) audio for XR applications. 6DOF applications are an opportunity for engineers and artists to present audio information in new and deeply immersive ways, and the chapter will look at practical approaches to the production of music and sound in this arena. The discussion encapsulates the thoughts and opinions of leading exponents in the videogames, music and entertainment industries, and provides practical information on creating high-quality immersive music and sound for ‘XR’ applications. The chapter provides a theoretical framework to understand the technologies and workflows involved in producing XR audio, a description of the hardware and software commonly being used, along with descriptions of the best working practices of leading producers. The discussion will look at both the limitations and opportunities offered by these new technologies, and will seek to clarify the landscape and provide a practical grounding for engineers working in the field. This chapter will talk to producers of procedural and adaptive music about music system design and will look at how music can be composed, engineered and deployed in XR environments. It will look at the new creative possibilities engendered by a 6DOF audio approach to music, and will examine the ways in which this is being currently deployed. The chapter will examine what is happening in research labs and at the cutting-edge of 6DOF art and technology, and will offer some suggestions as to the future landscape and economic outlook for these technologies