Can you believe your ears?

I love when I am working on a project that involves binaural sound with someone who hasn’t heard binaural sound before. Handing them a pair of headphones, playing them something and seeing them physically react by touching their head to make sure that the invisible electric razor buzzing near their ears isn’t really giving them a new hairstyle or turning, confused, to find the person speaking behind them and then bursting into surprised laughter when they realise there is no one there. They always seem to have that moment of confirmation, that visual or tactile check to make sure that what they are experiencing is an illusion and nothing more, and that they shouldn’t, though they are inclined to, believe their ears after all.

Recorded using two microphones in the ears of a real person or a fake ‘binaural head’ – that is, a fake human head shaped out of wood or another material – and played back through headphones, binaural sound replicates the way our ears really hear and thus allows our minds to spatially orientate the sound in relation to the listener. For example, a bee buzzes around your head, a car whizzes by, someone whispers into your ear causing your skin to prickle. It’s more than recorded sound, somehow. It feels real and live, like it’s happening around you in the physical world.

Aaron Hussain recording with Binaural Head he built for Sleight of Hand Theatre (Photo by Ellie Chadwick)

 In my work as a theatre practitioner experimenting with form and technologies, I’ve long been fascinated with theatrical uses of binaural sound. Relatively easy to produce, and even easier for audiences to access (even in a pandemic as all you need is a pair of headphones), binaural sound is arguably the simplest yet most effective form of technological immersion out there.

 But what immediately comes to mind if someone says ‘immersive technologies’? Perhaps Virtual or Augmented Reality, complex motion capture set-ups, or haptic feedback gloves. While these are all exciting tools that can create incredibly immersive experiences, leading us into another world and altering our sense of reality for a time, binaural sound’s ability to trick the brain into believing it is hearing real-time, three-dimensional, live sound is – I would say – just as effective at producing a powerful sense of alternate reality for audiences. Potentially even more effective, since it relies largely on the power of our own brains and imaginations in order to ‘fill the gaps’, and as we know from genres such as Horror Film, what isn’t shown is often more powerful (in Horror’s case, scarier) than what we see. Also, as neurologist Seth Horowitz points out, ‘one of the most important drivers of emotion is sound. And the reason it’s so important is because it works underneath our cognitive radar.’

 This kind of unconscious stimulation combined with the power to imagine makes for a powerful immersive tool. The power of imagination is, of course, intrinsic to the theatre, where audience and performer take part in a ritual contract whereby disbelief is suspended for a while and they buy into another reality. But in binaural sound, the other reality is arguably more than a conscious suspense of disbelief, it’s a deeper illusion that plays tricks on your mind. In its most effective moments, it can make the listener react completely naturally and instinctively to a given stimulation.

 In a performance I attended of David Rosenberg and Glenn Neath’s Ring at Battersea Arts Centre in 2014 (a piece set entirely in the dark, using binaural sound throughout) there was a moment where a character in the recording whispered in the collective ear of the audience, and several audience members screamed in genuine surprise. The effect was so strong that it was as if you could physically feel the character’s breath on your neck. The audience’s sense of truth was so distorted that such a moment felt entirely real.here was only one way to be separated from this new sense of reality and that was to remove your headphones and confirm, or realise, that there was no one there whispering or lurking in the darkness.

The physical nature of binaural sound is arguably what makes it so effective an illusion. In my academic research, I term this sensation ‘audiosomatic’, in that the technology is able to produce a reaction in the body that is highly physical. In fact, there are hundreds of videos online using binaural sound to aid relaxation by inducing a physical reaction; an autonomous sensory meridian response (ASMR), the experience of a static-like tingling sensation on the skin, often the back of the neck.

Since the work of creating the illusion is completed in the brain itself, utilising the way that we experience sound in everyday life, binaural becomes a particularly powerful tool in relation to the idea of creating a sense of ‘truth’ in fake perception i.e., in storytelling. Possibly this is why it is being used more frequently by experimental theatre practitioners.

Of course, during the pandemic, binaural and spatial sound are great tools for immersing audiences in stories while in their own homes. Various theatre companies and makers have been utilising binaural, often alongside other digital content, to reach audiences at a distance, such as Limbik Theatre with their audio-visual piece The Garden, Darkfield with their Darkfield Radio strand of work, and my company Sleight of Hand with projects Ergo Sum (an interactive experience about neurodiversity and mental health) and Silhouettes (an R&D project about the life and work of Ana Mendieta).

Oliver Church in Ergo Sum (Photo by Thom Buttery)

But while the technology lends itself well to these recent circumstances, it is not primarily the pandemic and social distancing that has provoked the desire to utilise binaural sound in theatrical storytelling. Our Ergo Sum online project has been adapted from a theatre show where audiences wore headphones and experienced haptic sensations and projection-mapped visuals to tell the characters’ stories in a live auditorium with performers. Darkfield, founded by Ring creators Rosenberg and Neath, began with binaural experiences in dark theatre spaces.

A particularly lauded example of binaural theatre from pre-Covid is The Encounter (2016) by Complicité. This one-man show takes audience members on an incredible journey through the Amazon rainforest using a combination of live and recorded binaural sound. The effect of the sound design on the imagination of the audience members is so visceral and ‘real’-feeling, that a family member of mine who had experienced the performance early in its run was surprised some months later when coming across the trailer online, since in her mind’s eye she had remembered the set as having trees and being much more naturalistic than a bare stage littered with wires. She had visualised a depiction of the Amazonian forest setting so vividly that she had half-forgotten it was created primarily through the immersive sound design and the power of imagination.

There was a similarly vivid impact on the imagination created by Ring, in which, as mentioned above, listeners hear a story told in binaural while sitting in complete darkness and wearing headphones. The audience sat in several rows on either side of the room, facing each other across a traverse stage. A character with crutches welcomed us in before the lights went down, his live audio melding seamlessly into a binaural recording which felt like a continuation of the live sound; the noise of his crutches walking around the room, the live rustling of the other audience members blending perfectly into a recording of the same. Once it was dark, the character asked us to rearrange our chairs.

Some nervous laughter was heard as everyone considered how to do this in the darkness. Then, everyone began to tentatively move around, chairs scraping and people negotiating obstacles in the dark, and a whisper in my ear told me that I was ok and I could stay put. The sounds around me revealed that other audience members had managed to move into a circle and they began talking, and the story commenced from there.

Of course, none of this was really happening. In reality, each audience member heard the same sounds, each person in the room was told they could stay where they were, and each one felt the circle being arranged around them. In my imagination, however, the other real-life audience members became the characters in the story. The whole room was rearranged and I was convincingly transported into a parallel, yet totally different, space and reality. It was so convincing that the belief manifested itself viscerally, in physical sensation. There were moments that sent a tingling shiver down my spine, and other times where the audio and utterly-dark environment led to the feeling of being able to almost see something: an experience verging on visual hallucination.

It is likely due to this illusionistic thrill and the way that binaural taps into the power of imagination that theatrical and artistic ventures utilising binaural technology are growing in number. They move beyond theatrical buildings, as well, into the streets. For example, Rosenberg’s Monument (2015) was an immersive piece taking place in London’s streets, combining binaural with live action integrating with the hubbub of the general public around you. Lavinia Greenlaw’s Audio Obscura (2011) took place at Kings Cross St Pancras, where audience members would wear headphones while wandering around the train station, hearing stories in the crowds.

Reviewer Kirsty McQuire reported that the experience of Audio Obscura was ‘like a kind of hypersensitive trance, induced by […] atmospheric sound’, saying that it took a while to come out of this heightened state of mind fostered by the work. McQuire also commented that the piece encouraged the projection of empathy onto the scenes witnessed, as you imagined the real-life people around you being the ones whose stories you could hear snatches of. Again, each audience member felt themselves placed more at the centre of the experience than is perhaps usual; an experience that was in large part created by their own interpretation of and response to the sensory stimuli around them. 

In 2019, The National Theatre gave binaural a try with the innovative show Anna, a tense political thriller where audiences wore headphones which gave them intimate access to the scenes, hearing from the character Anna’s perspective. Whispered conversations, overhearing snatches of information, and moving auditorily from room to room in the set enhanced the thriller experience by revealing information piece by piece. This gave a sense of intimacy and the sense of experiencing the storyline’s ups and downs from the perspective of the character at the centre of it all. In a similar manner, our production of Ergo Sum used binaural to give audiences a sense of being “inside the heads” of neurodiverse characters, including one who experiences autistic sensory overload, another struggling with flashbacks and trauma, and one who is having schizophrenic auditory hallucinations. By presenting the world from their perspective, the show promotes a sense of intimacy with the narratives and helps foster empathy and understanding.

As immersive technologies and experiences continue to grow in popularity, in a time when people are feeling more and more disconnected from each other with a lack of community in many areas, the polarising effects of social media, and recently social distancing taking its toll on collective mental health, it makes sense that audiences are gravitating towards illusory sensorial experiences that have the ability to transport and reconnect.

Whether sound or smell or touch, we relate to the sensorial on a deep and personal level.sensorially-immersive experience which works underneath our ‘cognitive radar’ encourages a particularly distinct feeling of emotional truth and an instinctive understanding, which – illusion or not – can impact us in much more powerful, personal, and memorable ways than any intellectual comprehension or logical truth. The memories of the Amazonian rainforest from The Encounter, of whispers and the moving chair circle in Ring, and of neurodiverse perspectives of the world in Ergo Sum, all are stored in the body via sensorial stimulation and so have a particular resonance or ‘ring of truth’ to them. 

In a way, there is little choice but to believe your ears with binaural sound, since its very workings depend on an illusion produced in your own brain. Even hallucinations have their own reality and in the performance of Ring on some level the chairs do move.

Examining immersive tools such as this lead us to consider new questions around the oft-discussed topic of the nature of truth when it comes to theatrical performance and storytelling. When does an illusion, representation or simulation of something become as, or more, real-feeling than actuality? What is the benefit of such experiences? What real effects, such as true empathy or a heightened emotion, can synthetic or pretend environments and experiences produce?

Recent developments in the field of cognitive science discuss the idea that narrative does not only represent but also constitutes reality. We can access truth and a sense of realness only through narrative understanding and, therefore, the act of understanding itself is our reality.  So, when the boundary between illusion and reality becomes blurred within the context of a story, as the popularity and power of theatre itself attests to, this is arguably when deeper understanding of a subject can be nurtured, and discoveries about the self and our emotional truth (both individual and collective) can be made; all via the embracing of a constructed environment and a good illusion.