Creating immersive audio stories for people with Parkinson’s disease

I have worked as a dramaturg and theatre maker for over 20 years, spending my life thinking about how to create stories for  audiences to lose themselves in. My professional toolbox is full of ways to develop the dramaturgy of an experience. Yet, in my new research project, I have had to put most of my previous experience and usual tools aside.

For 10 years, I witnessed my father’s experience with Parkinson’s disease (PD) and felt  the helplessness that comes with a condition with no known cure. I came across a chapter in a book which mentioned that mental imagery, and in particular motor imagery, was being explored as a tool to address common issues, such as freezing gait that people with PD experience. It happens when the brain’s messaging system to the body is disrupted and it leads to people grounding to a halt. Through engaging their mental motor imagery, patients can work around these disrupted neural connections and get their body moving again.  

At the time, I was working on a piece of immersive audio theatre called Reassembled Slightly Askew (RSA)[1] by Shannon Yee, which utilised binaural audio to place the audience within the body of Shannon as she falls ill with a rare brain infection, experiences the surgery and travels the long road to recovery with an acquired brain injury. My work as a dramaturg in this project was focused on how we helped our audiences create the right mental imagery for the story by combining text and sound design. The impact of the final production on its audience was powerful and very visceral and it sparked an idea for me in this new context.

The combination of the chapter on Parkinson’s and mental imagery and the experience of working with immersive audio made me feel that maybe I could contribute towards making my father’s life a bit better after all. I started to explore the possibility of a project using immersive audio for people like Dad. I wanted to create stories that transport them away from the lived reality of the illness, whilst engaging their motor imagery in the ways suggested by the research to keep the messaging routes open and active in their mind and bodies. 

Stephen Beggs as the nurse looking after audiences at Reassembled Slightly Askew UK tour 2017. Credit: Photo Michael Mutch, Northern Echo

Years later, this grew into my practice-based PhD project Worldbuilders and the research question: Can artistically developed immersive audio stories be a tool for engaging people, living with Parkinson’s disease, in developing their mental imagery skills whilst being transported by a story? Now, I am experimenting with different combinations of what we can learn from ongoing research using mental and motor imagery for therapy and rehabilitation, anecdotal evidence from people living with Parkinson’s and the extraordinary ways in which binaural sound communicates with our mind within immersive audio stories.

There are many aspects to consider when creating these experiences; audio techniques, immersive world-building, technologies and delivery and facilitation of the experience that fit within the arts and health field. However, in this article I will focus on some key learning points and avenues I am exploring from a writer’s perspective.

 Characters to narrator

The ‘stage space’ and the fictional world in immersive audio is inside the mind of the audience member who in turn is located at the centre of that world. One of the first major things to consider was mode of address. Who am I to the listener? How will the narrator address them? Where am I? Who are they in the imaginary world? What are the relationships between character, narrator and me as the writer? To make immersive audio fully effective, the audience needs to listen through headphones. The narrator’s voice is therefore placed snugly, and intimately, in the listeners ears. It required a change to my normal workflow, iteratively writing, recording and listening. I paid close attention to what kind of mental imagery the different mode of address created when heard through headphones. After several experiments exploring 1st person and 2nd person perspective, with some unfortunate accidents where I ended up with empty bodies moving around my stories with no subjective agency, I settled on 2nd person as the mode of address. In the headphones that felt right.  

The next step was to look at narrative distance and tense. I learnt it is incredibly easy to slip into the habits belonging to writing in 1st and 3rd person even when deliberately setting out to write 2nd person. I needed to anchor my listener/participant within the fictional immersive world in the here and now. I needed a very present sense of presence. However, I still needed to clarify who the ‘you’ is that is being addressed.

 Being yourself, vs being someone else

In Reassembled Slightly Askew, as in many other immersive experiences, stories and games, we were asking audience members to inhabit the body of someone else. In some immersive stories this could be either a fictional character or one based on someone’s experience. In RSA, our audience was in the body and experience of Shannon, and the sonic world was created from her perspective to share the particular sensory experience of what happened to Shannon. As an audience, you were not asked to do anything but perceive, listen, feel and imagine being Shannon.

However, that was not my task in hand. My task was to create bespoke audio stories for people whose motivation is primarily to see what the experience can do for them in the context of living with a progressive illness. It was clear that the main difference in Worldbuilders is that my audiences enter the experience as themselves. From the medical research into mental imagery for Parkinson’s, I knew that my audience had to imagine their own bodies within the story being physiologically active and reactive within the immersive world through all the senses, but most importantly proprioceptively and kinaesthetically.

Therefore, I needed to connect my audiences’ own motivations for participating in the experience, their physiological self as well as their own subjectively felt dramatic stakes: their illness. The stories needed 1st person pov (point of view) sensory and motor imagery detail and instructions to work in conjunction with the 2nd person pov narration. A crucial element to achieve this was to fold sensory and motor imagery and instruction into the stories but without losing pace and flow. I had to let go of my ways of working from theatre and create a new type of dramaturgy for Worldbuilders, which would facilitate the participant’s motivations, stakes and perspectives within the fictional, sonic world I was creating whilst giving them hidden tasks to do with their minds and bodies. I began exploring the dramaturgical requirements of co-creation. 

 Co-creation

To create a narrative for co-creation, I needed to provide story details such as a location, an action and a loosely set task, while leaving enough space for the listener to make their own decisions about what the task means to them. Addressing my listener as ‘you’ proved very helpful and I found that the listener needed more room to make decisions than in prose writing, but less so than in stage writing.

I ran into challenges all the time. Creating backstory without too much specific detail yet carrying emotional story impact was tricky. When working in an arts and health project, it is key to avoid inducing heightened adrenaline states. This wipes out 80% of what we know about dramatic stakes. 

Another hurdle was being able to create scenes where other characters are speaking to you – the listener/protagonist – without your feeling the need to answer.  If you record a voice that is representing your participant’s voice and thoughts, you immediately disrupt the hard-won subjective sense of self. The recorded voices’ accent, age, gender may get in the way for the participant in their task to maintain a sense of presence. I gave this function to the narrator which was an enjoyable experiment but quite limiting.

When mixing the audio, the key was to think through where within the sonic world to place certain sounds; the narrator versus any internal character noises or thoughts; where in the sonic space to place present tense emotions, any memories or thoughts? I am also experimenting with creating motor-focused sounds. With the aim to catalyse their feeling the space and their own movements within it through sounds rather than solely relying on words and the narrator.

A visual representation of the spatialization of different sounds within the sonic world.  Credit: created by Hanna Slattne in Miro using graphics by Andre from Noun Project.

The big question remains; can immersive audio stories transport the listener in the way that novels, plays and screen stories do while simultaneously asking them to work intensely with their mind/body connection within an immersive imaginary world? I am still not sure either way yet. However, I am learning a lot about audio story dramaturgy in the process and am trying things I would never have done had I not given myself this challenge. 

In the next part of my research, I am running consultations with people living with Parkinson’s disease to hear more about how to aid them in connecting their mind and body, how to tap into the memories of moving freely and to incorporate that into stories and experiences. In a way, I am asking them to co-create with themselves – with their minds, memories and imaginations. Is it possible to build up motor imagery skills from the comfort of your own armchair, without it feeling like a clinical exercise? Might working with your imagination in this way, genuinely enhance a sense of wellbeing? Most importantly, can we develop practical and usable tools to help them through the moments when freezing gait stops them in their track?  All while escaping from the world for a brief moment, through the magic of audio and story.

If you are interested in these questions and ideas, I would love to hear from you. Please contact hslattne01@qub.ac.uk 

[1] Reassembled Slightly Askew was written by Shannon Yee and created in collaboration with director Anna Newell, sound designer Paul Stapleton, Choreographer Stevie Prickett and dramaturg Hanna Slattne. For more information visit: http://reassembled.co.uk