Storytelling in 360° Sound: Audio Orchestrator

BBC Research and Development is the home of technological research and broadcast innovation at the BBC – behind innovations from Ceefax right up to modern day standards like UHD. One of the ways we continue to innovate is by experimenting with and exploring storytelling formats; from the thousand outcomes of branching narratives to exploring other worlds in AR and, of course, the immersion of audio storytelling. One of the most recent innovations from the BBC R&D Audio Team has been the development of Audio Orchestrator — a tool that allows users to create immersive, interactive audio experiences that play out through multiple connected devices. Audiences then use their everyday devices (from laptops and tablets to smartphones) to listen to a surround sound experience. Creators can use the tool to craft exciting surround sound experience that envelops the listener using just the devices that they have to hand.

Over the past year alone we have used Audio Orchestrator to put audiences into haunted houses, pitch-side for rugby matches, and at the conductor’s podium. Spatial audio — especially across orchestrated devices — has been a powerful tool in helping audiences to reconnect with creators whose normal mediums were not viable in 2020. We thought we’d share some of the things we’ve learnt telling stories using Audio Orchestrator over the past few years, from immersive horror to recreating stage plays.

Artwork for Audio Orchestrator

We have seen some amazing uses of spatial audio spring up in response to the ongoing restrictions on in-person events — proving the old adage that necessity is often the mother of invention. Last year, we partnered with theatre company 1927 to turn their stage-to-radio translation of Decameron Nights into a spatial audio experience using Audio Orchestrator. 1927 embraced the full capabilities of the tool, using the fact that audiences were creating their own listening environment (including instructions to turn down their lights) and used spatial audio to transform their existing radio play. The use of 360° audio gave the company much more scope to use techniques more appropriate to the theatre, such as choreographing entrances and exits from different devices and angles, and giving their radio play three dimensions by adding directionality to individual characters and sounds.

We also saw the spatial audio used to give sport fans a reminder of experiencing live events, letting viewers customise their own audio for Six Nations highlights – from referee mics, to surrounding themselves with crowd noises as they would in the stadium. The vast potential of Audio Orchestrator means that creators can play with narrative and audience experiences in many ways. From giving individual audience members different parts of the story and asking them to solve a mystery, to using a connected mobile device to act like a phone during the experience, or asking people to hide devices for others to find, there are hundreds of potential applications.

A promotional image for Audio Orchestrator experience Monster

The ability to physically place sound anywhere can be a powerful tool for horror storytelling; you can give the classic jump scare even more power or give audiences the very real sense that something is breathing down their neck. This is particularly powerful with connected devices as you can instruct users where to place their devices for maximum impact. Monster, which you can learn more about on BBC Taster, used connected devices to tell a ghost story. The writer Brad Birch talked about the power of the technology to heighten horror’s ability to imbue every day spaces with a sense of dread. Creators can also cater their experiences to the at-home listening environments — for example turning the listener’s home into a haunted houses, or only using auxiliary devices for jumps scares. Other, non-Audio Orchestrator, binaural experiences have encouraged listeners to move between rooms to avoid detection (such as in BBC Arts commission Hairy Hands FM) or turned their local park into a plague infected wasteland (such as in popular mobile app Zombies, Run!). Spatial audio can be a hugely effective tool in scaring audiences and immersing them in horror – and Audio Orchestrator allows creators to change the very environment around the listener.

We would really encourage you to explore creating with spatial audio in all its forms — from Audio Orchestrator to other tools like the EAR production suite. Spatial Audio is a unique and exciting way to capture and hold your audience’s attention. Its multiple applications across genres and forms make it ideal for engaging audiences, and exploring the boundaries of storytelling. You can try out Audio Orchestrator, and our other storytelling tools,  for free on MakerBox now. And we’d love to hear about your experiments and ideas with the tool, so why not drop us a line or ask us a question in the MakerBox Community Forum or get in touch with us at makerbox@bbc.co.uk?