Live theatre performance in XR (extended reality), which includes both Virtual Reality (VR) and Augmented Reality (AR), is a rapidly growing field of digital drama that sits at the convergence of immersive theatre and interactive games design. While a number of theatre companies have adapted their existing work for VR, (notably the Royal Shakespeare Company with Dream Online in March 2021), our research project, Mrs Nemo XR, aims to create new works of musical theatre specifically written, designed and built to utilise the interactive and immersive features of this exciting new performance medium.

Mrs Nemo XR

Our initial project, a short-form immersive musical — which we term an ‘Immersical®’— is inspired by an episode from Jules Verne’s classic Victorian adventure story, Twenty Thousand Leagues under the Sea (Verne, 1, featuring the attack of Captain Nemo’s Nautilus submarine by a giant sea creature, the mythical ‘Kraken’. Adding a contemporary twist to the tale, the story is retold from the perspective of a mysterious female character in a Victorian bath chair, who appears at the outset to be Nemo’s wife, but who is later revealed to be a mermaid, one of the many marine creatures that Captain Nemo has collected for the undersea museum which we use as the setting for the show. 

Although we draw on Verne’s original story for the undersea environment and basic plotline, we have devised our own narrative to tell the tale from an alternate viewpoint; that of one of the sea creatures whom Nemo has captured. As the audience members join the show, they find themselves embodied as individual avatars of deep-sea divers, who have been transported into the submarine, and invited by the curator (Mrs Nemo) to explore the gallery displaying the museum artifacts. As Mrs Nemo tells her own tale through improvised dialogue and song, the audience can interact with the actor by talking to her avatar and examining in 3D some of the items in the museum. 

VR Platform

Staged on the browser-based online social VR platform Mozilla Hubs, accessible  either via desktop computer or VR Headset, Mrs Nemo XR is performed by a solo actor working remotely from home, singing live to pre-recorded music tracks. We are limited to one live singer, due to audio latency issues across the network which prevent multiple performers singing in sync. The show also has a short run-time of ten minutes due to concerns of how prolonged use of VR may adversely affect actors and audience members, especially new users of the technology.

Performances 

Our initial trial run of twelve performances with a live audience began in October 2021 and was staged in collaboration with an informal collective of virtual theatre-makers from around the world under the title OnBoard XR. While the whole event shared an underwater-themed virtual environment, each team designed their own show, including the set, props, avatars, and individual backstage cueing system. This article details the technical, design and production challenges we encountered while making Mrs Nemo XR, with specific reference to the narrative design and scenography of the production.

Performer singing in a VR Headset, while moving her own avatar in VR

Creative Team

This virtual theatre production, as with a typical physical theatre company, has a creative team composed of Producer, Writer, Director, Designer, 3D Artist and Performer; all theatre practitioners and creative technologists, several of us conducting research at Digital Creativity Labs, at the University of York, UK. Throughout the four-week production period in the autumn of 2021, team members worked remotely from different locations across the UK and the USA. Pre-production meetings, rehearsals and performances all took place online, managed through a combination of VR, video conferencing and text and audio messaging. Many of the sessions were intense early morning or late-night events, due to the complexity of distance working across time zones.

Interactive Narrative Design

Mrs Nemo XR, aims to engage the audience with the narrative, while giving them the freedom to explore the immersive submarine environment and interact with the performer and each other. Without a traditional framing device, such as a proscenium arch or cinema screen, we rely on alternate ways of directing audience focus, such as guided direction and environmental storytelling. Our audience, who were represented by avatars of deep-sea divers, were actively encouraged to be part of the story-world, through improvised dialogue and narrative song, and the way in which the actor directed their attention to the interactive props, such as the toy Kraken that could be picked up, moved and resized, and scenic effects like the sudden and dramatic appearance of the giant sea monster outside the viewing window of the sub.

Technical Challenges

One of the main technical issues we encountered when using Mozilla Hubs with our Meta Quest 2 (previously the Oculus Quest 2 VR headsets), is the limited audio capability afforded by this form of Web-based VR (a VR platform that is easily accessible on multiple devices via the Internet). While we did have the capability for spatial audio, we elected instead to have our performer’s voice override this feature, so that she could be clearly audible, regardless of position in relation to the listener. Since we knew that audio and visual latency would be problematic, our actor ran the music tracks from her laptop, whilst performing in a VR Headset and manipulating her own avatar, thus syncing all her actions at the point of transmission. 

The scenic cues, including the change of avatar for Mrs Nemo for the final reveal that she is a mermaid, and the attack of the Kraken, were also subject to delay of operation across the internet, and so were run remotely by the backstage crew to avoid distracting the performer by the desynchronised scene changes. This also enabled the production team to make live adjustments during the show when latency of operation necessitated the removal of used props from previous scenes or by ‘respawning’ the placement of set pieces that had entered the scene at the incorrect size or location.  

Scenography 

The inspiration for set design, props, avatars, and interactive artifacts were both derived from Verne’s original story, and from the writer’s scripted dialogue and lyrics. From the book it was clear that while the Nautilus did have a library and salon with a large viewing window, there was no dedicated space for storing or displaying the Captain’s undersea collection. This led to the idea that the salon should become a museum, its walls filled with books and artifacts relevant to the story. Using Mozilla Hubs’ custom editor, Spoke, which allows developers to create 3D worlds with their own designs or pre-made assets, we built the submarine interior, and using a bespoke plugin Stage Management System, we were able to program cues to make scenic objects, props and avatars appear, animate, and change size and position within the scene. 

Tech rehearsals in Zoom and Hubs showing the cues of the stage management system

Working in Mozilla Hubs

In order to perform well across a wide-range of devices, Hubs has certain operating limitations, such as the inability to move or ‘spawn’ objects easily, and a maximum scene size of 16MB, which resulted in our immersical taking on a low-poly (an art style where the number of polygons in a 3D model are reduced to give a low quality appearance to the resulting graphics. This style has the benefit of being efficient and easy to optimise)

style of graphics. Because of internet latency, we could not accurately predict how the scene would appear on the various viewing devices that would be subject to differing connections, internet speeds and locations. For example, we aimed to trigger the Kraken’s appearance outside the viewing window, as a shocking reveal to the audience, especially for those in VR who would see a gigantic monster attach itself to the side of the submarine. However,  the physical distance between the stage manager in York (UK), the performer in New York (US) and the worldwide audience, meant that the significant delay between cue and action led to some audience members, depending on the efficacy of their own devices and connection, experiencing a slight delay between what the actor was saying and what the scene was showing. 

Set Design and Layout

Our initial design plan for Mrs Nemo XR was to divide the submarine into smaller clustered areas to direct audience movement, (see Figure 3) to where the action in the scene was taking place. We particularly wanted to guide the audience towards the window to provide a good view of the Kraken’s sudden appearance outside the submarine. During rehearsals however, we realised that the audience tended to cluster around the performer and follow her around the scene. This led us to strip back some of the central blocked-out areas to provide a clear line of sight throughout the submarine interior, so that the performer and the action could be seen at all times.

Preliminary design of the submarine interior

Sub interior with Captain Nemo’s portrait on the wall

The eye of the Kraken at the window

Audience member as diver with props

Conclusion 

In spite of the technical limitations of the Mozilla Hubs platform, in particular the external cueing system, the low-poly graphics, and the audio/visual latency experienced, the show was well received and we felt we had achieved our central aim of creating our first fully functioning immersical. Through the rehearsal process and by observing the performances, we established methods of directing audience focus and promoting their engagement with the narrative, in three key non-verbal ways. Firstly, through the use of scenic devices like the viewing window, to frame the attack of the Kraken, and secondly, in spite of internet latency, we were able to sync actions to the music, cueing movement of scenery and interactive props at appropriate times. Thirdly, we found that  guided direction from the performer (e.g. gestures, movement, following her gaze and use of interactive props), supported audience attention and engagement with the story-world. 

Our research continues with a new iteration of the show, where we will be using a different VR platform, and extending the length of the performance, working with innovative techniques for reducing audio latency. We also intend to offer greater user agency and interactivity, and explore different styles of narrative design including non-linear and object-based storytelling. We see a great future for live performance in XR, particularly in terms of ease of access and affordability, and look forward to our next collaboration within the rapidly expanding community of VR theatre-makers and their virtual audiences.

Bibliography

RSC Dream online. Audience of the Future Live. (n.d.). Dream. [online]
RSC Dream online. Latest Press Releases | Royal Shakespeare Company.
Mrs Nemo XR – performance video #OnBoardXR [SHOW3.3] Live Short Series in WebVR.
Verne, J. Twenty Thousand Leagues Under the Sea. (first published in English, 1872). Project Gutenberg.

Credits

Mrs Nemo XR Creative Team: Director: David Gochfeld, Writer: Mary Stewart-David, Designer: Daniel Lock, Exec Producer: Cristobal Catalan, 3D Artist: Guy Schofield, Performer: Vivian Belosky
OnBoard Stage Management System developers: Roman Miletitch, David Gochfeld, Clemence Debaig, Michael Morran.
Digital Creativity Labs & Department of Theatre, Film, TV and Interactive Media, University of York

Have you ever felt your skin prick up as you walked into a foreign environment? Throughout my life there have been times where I’ve entered an unknown venue, be it when exploring the world as a solo traveller, or starting a new job. The mixture of excitement and anxiety that I’ve associated with an entry into the unknown has been a full body experience, reflected in the pores of my skin and in the heat pulsing through my limbs.

I reflected on this phenomenon after my somewhat recent experience of Tender Claws’ The Under Presents, which is described by its makers as ‘an intriguing multiplayer experience set between two worlds’. Others might describe it as a clever and mysterious VR game that affords the user six degrees of freedom.

Image from: https://tenderclaws.com/theunderpresents

First activating the free demo version of this experience in the Oculus Quest, I found myself in a dark and waterlogged wasteland, with little idea of what to expect. I edged towards the ruins of a nightclub from which muffled music exuded. After several moments of uncertainty, a mysterious guide appeared and shepherded me into the building. I was followed into a room full of doors by masked figures in black capes. After some moments, I realised that their strange actions were actually a replay of my own awkward explorations of the space moments earlier. Past and present collided, giving me an uncanny insight into my own behaviour. As with the ‘real world’ experience described above, I felt acutely aware of my own body in this unknown virtual environment.

After much experimentation, I managed to access the depths of the nightclub at the heart of The Under Presents, where a number of other black hooded spectators milled about, enjoying the on-stage entertainment. Are these real people like me, I wondered? It certainly seemed a possibility. Like me, the other guests clumsily engaged in creating and eating food while exploring the environment. One hooded figure handed me a keyboard guitar before scurrying towards an exit.

Image from: https://tenderclaws.com/theunderpresents

Unfortunately for me, I came to this VR experience somewhat late. It’s notable that earlier iterations of The Under Presents saw live actors teleported into the multiplayer experience to interact with users in what could be described as virtual live immersive theatre. A quick scan of the reviews of the project on the Oculus website reveals the novelty of this spontaneous and communal experience. A video on Tender Claw’s website outlines the innovative interface that allows actors to enter into the VR experience, choosing their appearance and powers that they can utilise to interact with players, so as to enhance the narrative experience. 

The affective nature of video games featuring navigable 3D environments has been explored by a number of scholars (Crick, 2011; Klevjer, 2012; Scriven, 2018). This comes after significant work interrogating the way that cinema ‘moves’ and ‘touches’ us on a bodily level (Marks, 1996; Shaviro, 1994; Sobchack, 2004; Williams, 1999). Klevjer describes how video games extend our bodies into the screen space:

When we play, because the avatar extends the body rather than pure agency or subjectivity, screen space becomes a world that we are subjected to, a place we inhabit and where we struggle for survival. We learn to intuitively judge, like we do in the real world, the opportunities and dangers of the environment. Klevjer, 2012, p. 13

This quote resonates with my experience of The Under Presents, where I slowly learned the rules of the virtual world, thus enabling me to progress through the experience. In doing so, I had to embrace ‘unknowing’, meaning I had to come to terms with my vulnerability as a virtual body in a foreign environment. I was never quite clear on where I was located or where I was headed, but in retrospect, this was half the fun. 

In recent years, emerging scholarship has probed the specificity of narrative VR experiences in terms of spectator positioning, presence and/or embodiment (for example, Dooley, 2021; Nicolae, 2018; Vosmeer & Schouten, 2017). The bulk of this work is concerned with cinematic virtual reality rather than more interactive VR experiences with narratives that are emergent or open-ended. I would suggest that The Under Presents offers a case study of a narrative experience that successfully draws upon immersive theatre and video game conventions, as well as cinematic tropes, giving rise to a unique embodied experience that is about ‘story making’ rather than ‘story telling’. The computer-generated story world provides a playground for multiplayer interaction, but the mystery of a stranded ship (the ‘time boat’) sits at the heart of the narrative experience. The limits of the game seem linked to the user’s tolerance for learning the rules of the world; in other words, there are seemingly endless threads to explore, if one takes the time to uncover them. 

My enjoyment of The Under Presents makes me think that the notion of embracing ‘unknowing’ is an apt one for VR creators as well as audiences. In the past, I’ve written about the need for creators to guide the viewer through a narrative VR experience, creating attention cues and allowing time for exploration and acclimatisation. As described above, the makers of The Under Presents adopted these techniques to some degree. However, I’d argue that the experience’s frequent lack of signposting and foreshadowing fosters a sense of uncertainty and intrigue that increases rather than decreases engagement. Maybe this is what great VR storytelling is all about – creating an affective experience that allows the user to ‘live’ the story while navigating unfamiliar territory. 

Postscript: The Under Presents is once again featuring live actors in the environment in the month of April 2022, so this is a good time to visit or revisit the experience!

References

Crick, T. 2011. ‘The game body: Toward a phenomenology of contemporary video gaming’. Games and Culture, 6(3), 259-269.

Dooley, K. 2021. Cinematic Virtual Reality: A Critical Study of 21st Century Approaches and Practices. Springer Nature.

Klevjer, R. 2012. ‘Enter the avatar: The phenomenology of prosthetic telepresence in computer games’. In The philosophy of computer games (pp. 17-38). Springer, Dordrecht.

Marks, L. U. 1996. The skin of the film: experimental cinema and intercultural experience. University of Rochester.

Nicolae, D. F. 2018. ‘Spectator Perspectives in Virtual Reality Cinematography. The Witness, the Hero and the Impersonator’. Ekphrasis. Images, Cinema, Theory, Media, 20(2), 168-180.

Scriven, P. 2018. ‘The phenomenology of the “other” in computer game worlds’. Games and Culture, 13(2), 193-210.

Shaviro, S. 1994. The cinematic body. U of Minnesota Press.

Sobchack, V. 2004. Carnal thoughts. University of California Press.

Vosmeer, M., & Schouten, B. 2017, June. ‘Project Orpheus: a research study into 360 cinematic VR’. In Proceedings of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video (pp. 85-90).

Williams, L. 1999. Hard Core: Power, Pleasure, and the” frenzy of the Visible”. Univ of California Press.

Dear metaverse diary, as I sit down to pour out my thoughts and try to find the exact place within my intricate mind that stores all my experiences related to live acting in virtual reality (VR), and, more specifically, my combined role as a VR actor, stage director and scholar within the XR theatre collective La Cuarta Pared VR [The Fourth Wall VR], the usual feeling that I have nothing interesting to say pervades my spirit. And it is exactly within this attempt to situate myself that I find some insight into this new art form and practice.

Acting is very much about finding your centre and situating yourself within a given physical and symbolic dramaturgic context. As a classically trained actor, I was taught how to become aware of my body and mind within the moment, being in the ‘zone’, feet on the ground, fully present. An actor’s presence on stage is related to a kind of expanded awareness that allows us to be extra attentive to our surroundings. It is as if an invisible pair of chameleon eyes pop out in our heads, allowing us to get an extra, almost 360 degree perception of everything and everyone on stage. Being present on stage involves sustaining attention moment by moment on all that is happening at once in the broader environment.

Ultimately, stage presence seems to be connected to this visceral, sweaty, physical realm where actors dwell and is apparently only accessible through a direct, strong sensorial experience that materialises in one’s imagination, thus fully realising the ‘theatrical act’. In other words, the mental, symbolic, emotional, and aesthetic construct that was meticulously elaborated on the actor’s mind is released into the physical world, an almost ‘cause and effect’ phenomenon. Thus, imagination takes place on the stage, whereas in VR one might say that the stage is imagination itself, as within VR the creative and expressive scenographic possibilities are potentially unlimited. One can craft magic, bend the laws of physics, fly, or even become a giant.

(…) both on-stage and VR presence stems from a desire for immediacy – being in the here and now. Both also involve a suspension of disbelief – a wilful denial of the artificial conventions of a performance or VR creation – on the part of the spectator or user (Samur, 2016).

In this sense, what happens to presence when an actor puts on a VR headset and is transported to a virtual environment? What changes perceptually as one embodies an avatar and, like a puppet master, gives life, breath and meaning to this digital skin? As I entered the world of acting in the ‘metaverse’ – today’s most hyped term, but somehow lacking a true critical approach to it – my first challenge was to learn how to embody my digital skin, my avatar. As a well-behaved actor, I looked for a place in the VR social platform VRChat where I could be alone with myself in front of a mirror. I was looking for familiar places that could keep me connected to real-world substance and my previous, non-VR experience as a theatre actor. I later realised that I could rehearse anywhere in the metaverse. I could rehearse while being suspended from the floor or even upside down. Later, I realised that my avatar could be anything; a floating orb of light or a cube. But my first instinct was to create an avatar that would resemble my real physical features. Yet, it was but a shadow, an impression, like a painter’s brush stroke depicting a landscape. Nevertheless, I was humanoid, and that was enough for me at that moment. Becoming native to a new art form takes its time. I wasn’t in a rush.

Me, myself, and my avatar

Non-metaversical impressions of meArtworks by artists Ana Pinho (left and right) and O Mimi Cha (at the centre)

Once inside my avatar, I needed to see how my digital body was responding to my commands. What were my limitations? This was an actor’s exercise, not a new user on social VR trying to get the grips with a new headset and controllers. This was an actor deeply engaged in understanding his place in this alternate reality. I was immediately impressed by the real time response and how refined these technologies had become since I first experimented with them back in 2015 when I started doing my PhD in Immersive Media. Back then, because all artists are chronically unsatisfied, I started to see certain flaws. My legs were acting weird, my movements weren’t exactly as subtle as I wanted them to be. My mindset was still outside, rooted in my non-VR theatre experience. I was trying desperately to emulate and bring my know-how as a performer from the ‘real reality’ to VR.

Actor’s warm up on VRChat

As I moved on into the different worlds on VRChat and started to experience flying or jumping from one mountain to another, I realised that the medium had its own unique nuances and that the apparent limitations of my avatar puppeteering were only limited by my own squared vision. And just as Plato legendarily banished poets from the ‘ideal city’ or Aristotle banished performance from any serious consideration of tragedy, this actor of yours started to allow himself to establish a symbolic, aesthetic, and performative encounter with the metaverse, free from conventions, trying to unveil its potentialities.

Let’s go full on cliché and make an analogy with the quote ‘To be or not to be!’ With live acting in VR, we can now both be and not be. I mean, when we act in VR we are physical body and flesh, and simultaneously a virtual entity (avatar). This questions not only the dramaturgic idea of the classical Hamletian dualism but also the actor’s work in VR. In other words, as I continued to experiment with my digital body I slowly allowed myself to be drenched with Samur’s (2016) ‘suspension of disbelief’ and it felt more as a merging process between my avatar and my real self than a mere manipulation of the digital skin by the performer. In other words, what occurs as one spends enough time in the metaverse, is a sort of avatar anthropomorphic process enabled by the illusion of immersion, presence, and plausibility that culminates with the appropriation of a virtual body. Nothing in between, almost like demonic possession.

As I started to create live performances with my colleagues and fellow artists at La Cuarta Pared VR [The Fourth Wall VR], I realised that the process was distinct. We were immediately drawn to building worlds instead of thinking about building characters and, ultimately, created characters to inhabit those storyworlds. At first, we were trying to situate ourselves and our performances within a place, a corner, a location in these worlds, as if the rest of the world was out of reach to us. In the real world, we can make use of the stage, or, sometimes, even the theatre building as whole, or other places. There are boundaries that we usually establish as performers in the ‘real reality’. In VR, we can use the entire world as we have created it. It was at the intersection of creating and performing in VR that my eureka moment occurred. I realised it is not about doing theatre in VR, it is about theatricalising these VR worlds. We can have parallel narratives occurring simultaneously and the audience doesn’t have to be compiled on a block somewhere, like a roman legion formation waiting for action. They can walk around these theatrical worlds and choose what to experience and whom to talk with.

These are not random social interactions in VR; this is performance, this is immersive theatre going into new heights as the audience becomes a participant within the stories. It is as Peter Brook’s famous idea of the theatre that occupies an empty space – ‘I can take any empty space and call it a bare stage’ (1969). As Brook says, ‘[a] man walks across this empty space while someone else is watching him, and this is all that is needed for an act of theatre to be engaged’ (1968). Perhaps this now needs rephrasing to ‘I can take a virtual empty space and call it a bare stage’. An avatar walks across this empty space while someone or something else is watching him. Narrative, actors, audience as participants, lucid dream and theatre become one fully formed whimsical experience accessible by anyone, anywhere. In a world dealing with a pandemic, entering and experiencing these storyworlds seems quite exhilarating.

The Council of the Titans teaser, an interactive live performance in VR by La Cuarta Pared VR [The Fourth Wall VR]

By taking the idea of ‘it is not about doing theatre in VR’ one step further we see that it is about doing theatre in VR, but also about creating worlds that are a construct of distinct languages and disciplines. Theatre in VR is a multi expressive art form at the intersection of hard science and digital humanities, drawing from areas as varied as art, narrative, anthropology, psychology, human-computer interaction and gaming. We call it ‘theatre in VR’ because we lack a better word for it, but it will come. With live acting in VR and other related emergent phenomena, we are reaching a unique moment in theatre and performance history. In it, we move from and beyond the conventional paradigm – where performance was mediated by technology – towards a new, multi-layered practice where technology is not the medium ‘in between’ but a means of instantaneous transportation to alternate realities permeated by theatrical and performative significance. It is obviously a debatable idea, yet through a symbolic, philosophical approach, it is coherent in this actor’s mind. Hakyung Sim (2021) takes this idea further by arguing that today we are at threshold of a new era, where ‘age-old theatrical’ performers and spectators’ give place to ‘posthuman alternatives’ pervaded by emergent digital tools and worlds that promise to revolutionise theatre praxis and experience:

Today’s theatre no longer solely depends on the bodies of live performers, and it increasingly draws on technological devices and digitally mediated networks to hint at potential posthuman alternatives to our age-old theatrical institution of live performers and spectators. In VR theatre, the prospect of posthuman spectators looms as the human body is aided by goggles or head-mounted displays (HMD) that function as prosthetic eyes. Digitally mediated images in theatre were previously considered to act as a Derridean supplement to the weak presence or complete absence of human bodies on stage, thus highlighting the prerequisite essentiality of live bodies. However, new technologies and media introduced in theatre emphasise the perception and bodily senses of spectators and how they, and not necessarily the live actors on stage, constitute the essence of performance’ (Sim, 2021).

In conclusion, dear diary, theatre seems to always find its way through obscure times. The recent rise in live performances and theatrical VR acts marks the beginning of a period of innovation, where theatre, performance and technologies come together to provide an escape from our everyday problems, allowing our imaginations to fly, literally. These words are loose thoughts on my journey as a live performer in the metaverse and as an interdisciplinary scholar trying to unveil the conceptual entanglements of this emergent art form. It is clear that thinking and creating within immersion is key to developing a native creative mindset. The big money questions are: how will digital performance evolve and be shaped by XR technologies within the metaverse, or the still infant idea of the metaverse, and what kind of challenges will there be for the theatre industry in the midst of  web 3.0, the embodied internet, NFTs, blockchain and digital real-estate? Where does a human-driven theatrical, aesthetic and affective substance fit within this ecosystem? Either utopian or dystopian, the future of theatre and performance is hybrid, there is no doubt about that. My final question is how far down the rabbit hole are we willing to go?

References

Brook, P. (1968) The empty space. Touchstone.

Samur, S. X. (2016) ‘Comparing Stage Presence and Virtual Reality Presence’, Revista Brasileira de Estudos da Presença, 6(2), pp. 242–265.

Sim, H. (2021) ‘Seeing Alone Yet Together: Modern-day Tiresias in VR theatre’, Performance Research, 26(1–2), pp. 69–77.

“Alexa, open The Messlins” says one of my kids out loud to the smart speaker setup in our living room. After eight months of research, design and production, I was about to witness my children interacting with my voice-enabled audio story for the very first time.

With a background in performance and media production, I have always been driven to find new ways to connect with an audience, whether taking the stage to do sketch comedy, giving live readings of my self-published children’s book, or producing audio-visual content for broadcast, film, and web. In 2019, I began to explore the ways in which storytelling can transform into storyliving; an undeniable shift that we have been seeing more of in the last decade. This exploration eventually led me to Ryerson University’s Media Production MA program. 

I completed my master’s degree in 2020 with a research and design focus on interactive audio narratives using a smart speaker. Interactive stories told through a smart speaker are a relatively new way to engage an audience. As part of my research, I developed a prototype for an interactive audio narrative for children titled “The Messlins,” using Amazon’s voice-enabled device, the Echo. The story used a branching narrative design. Branching narrative is a form of storytelling that allows audiences to decide the path and outcome of the story (gamasutra.com). A child can decide how the story unfolds by voicing their choice when prompted by the characters heard in the story. Later, I will describe further why the Amazon Echo was chosen as the device for the design and user-testing of the prototype.

As a parent, my world presently revolves around all things that educate and entertain my twin 6-year olds, so it made the most sense to write a story for an audience for that age group.  Through my research, I learned that children between the ages of 4 to 7 have developed the skills to use their imagination and play make-believe (Calvert and Wilson 291; Vygotsky 1967). Many of us can recall a time in our own childhoods when a parent reads a bedtime story. Even though my mother or father had the responsibility of reading the words on the page, I was always encouraged to ask questions, add sound effects or speak “directly” with the characters. The beautiful thing about an audio narrative is that it can open a world of possibility that allows the listener to paint their own imagery of the storyworld and characters in their minds. 

Since the prototype was designed to be an interactive experience, it was important to understand what motivates a young child to interact with the story. One consideration was a child’s ability to communicate verbally. Storytelling with elements of play can help a child with their communication skills (Phillips 4; Ryokai and Cassel 2). Throughout the story of “The Messlins”, children are meant to interact with the characters as well as move around their physical space as if they are in the story. The design of the characters was another important factor when considering a child’s motivation to interact with the story. The main characters are siblings who would be described as tweens, as it has been found that young children respond better to older characters that they can look up to (Miller 2014).  As a side note, the inspiration for the main characters with whom the user interacts came from the adventurous spirit of both my son and daughter, so I decided to create a young male and female character with equal status. I also wanted to pay a small tribute to my heritage as a Filipino-German-Canadian, so I created another main character with a German name while weaving in Filipino references throughout the script. 

My years of experience as a video producer taught me to always start with a script outline, a “blueprint” that ensures flow and connectivity between the beginning, middle and end. An interactive story with multiple plots and endings adds an extra layer of complexity. Before writing the audio script, I first mapped out the main plot points of the story which evolved into a branching map, which looks very much like a tree, hence the name. Unlike a linear plotline, stories that employ a branching narrative require a lot of work and pre-planning to ensure that every choice available makes sense for the story progression (Crawford 117). I personally like to get literally hands-on for a project like this, so to begin I created the branching map by hand with lots of different coloured markers and a large piece of paper; this is where being a parent with an abundance of craft material comes in handy! I then created a digital version of the branching map using a mapping software called Lucid Chart.

There is one differentiating factor that sets interactive content apart from a linear audio story; the ability to make choices. In a branching narrative, the listener can influence the direction of the story. A narrative designer must consider how a user will interact with the system and factor in as many outcomes as possible (Crawford 31). The element of play involved in an interactive experience also changes the role of the child. When a child simply listens to a linear story, this is considered a passive activity. However, in an interactive story a child is an active participant who can be viewed as either a player, user, learner, or a combination of the three (Markopoulos et al 28). Designing for an active participant requires empathising with them, in this case the child, and seeing the story through their eyes. In the case of my prototype, it would be hearing it through their ears. 

Once an idea is sparked, the hardest part of the creative design process for me is often simply starting. I was carrying out two roles in parallel; the writer of a fairytale story and the VUI designer of an interactive experience. The writer in me needed to think about how to tell an engaging story that hit all the major beats. If the experience wasn’t entertaining, I would lose the interest of the listener (Buurman 1). While my inner-VUI designer had to consider the conversation between the user and the device from beginning to end (Pearl 8).  For no particular reason, other than it being one of the most popular brands of smart speakers that I also happen to own, I chose the Amazon Echo, which uses the virtual assistant AI technology called Alexa. 

Voice-enabled apps that are built for the Echo are called “Skills.”  Early in the design process, I knew that I wanted a narrator to introduce the Skill before the story began. I also wanted it to be a voice that was different from the voice assistant (in this case, Alexa) so that the user would know that the experience had officially begun. This meant that the narration portion would have to be scripted and recorded. 

Once I had established how the Skill would launch, mapping out the branches was a relatively fluid process. I would complete one whole branch from start to finish before going onto another branch. Some branching narrative designers believe that it is best to follow through to the end with one branch otherwise one may run the risk of creating several half-complete branches (LudoNarraCon 2020). In addition, this approach insured that every path was given the same importance in the design process. To avoid the map from branching out exponentially, I used a technique referred to as foldback, which involves linking certain points of different branches together (Crawford 121). Once completed, the first version of the branching map resulted in a total of 31 scenes which included the opening narration and credits. 

With the map created, I was then able to write the script, a process which was once again twofold; the beginning and end of each scene were driven by principles of VUI design while the middle section was all about the narrative. The front and backend of each scene were treated as conversational markers (Pearl 40). This means that a scene opens with positive feedback to the user indicating that the choice they had made in the previous scene was heard and accepted. It also means that the same scene would eventually end with one of the characters prompting the user to make a choice for the next scene. I opted for the characters to suggest how to respond as it has been found that providing examples is a better user experience (Pearl 21). 

The original branching map included two distinct choices for the user at every major plot point. My original thought was that it was best to keep the answer short and simple as to make it easier for the child to repeat. I later discovered that short commands like “Study” can be problematic for a voice-driven application and it is easier for the system to recognise phrases such as “Go to the Study” (Pearl 131). Of course, we can’t forget about the middle section of each scene; this was my moment to shine as a storyteller and comedian. I was making a prototype for kids after all and humour can make an experience more enjoyable (Hall and Maeda 95) but it is important to note that it is different from adult humour (Miller 2014). Fortunately, I have a strong background in sketch comedy which makes me keenly aware of comedic timing and the art of being silly. I added little moments in the script that would hopefully elicit a giggle from the listener, whether it was for the child or their parent. 

When the university closed its doors due to the Covid-19 pandemic, I had to be proactive to ensure that the production process would continue as smoothly as possible. Unfortunately, the social distancing rule that was put in place in mid-March 2020 made in-person recording sessions impossible and I had to pivot my approach. I planned a virtual sound production that required building a temporary sound booth in my basement, delivering equipment to the homes of my voice talent, directing them over a video conference call, and uploading the audio files onto a shared drive which my sound engineer could access.

While I was able to find alternate methods to keep production going, it did not come without its challenges and setbacks. The sound booth that I was able to build at home was the perfect size but I was unable to cover it with blankets from top to bottom, which would have been ideal in order to cut out any external sounds. This meant that there were moments when noise in the home like the furnace going on, would get picked up by the microphone. Directing the voice talent remotely was also challenging. I had to listen to my voice actors through my computer speakers. There was no way of really knowing how the recordings sounded until I received the files. Fortunately, my voice actors also live together which made it easier to match their ambient sound. The recordings that my husband and I did at home also matched. Adjusting the reverb and adding ambient sounds created the illusion that the characters were in the same space. This is where having a sound engineer who can make this happen is incredibly important.

The audio clips were made up of various sound elements which included voice recordings, original and stock music, and sound effects. The voice recordings were provided by myself, my husband, and two friends. Since this was an interactive audio story for children, I wanted all the characters including the mischievous creatures, to be lively and likeable. The high energy delivery of the character voices was intended to keep a young listener engaged. As sound can affect us psychologically and emotionally (Treasure 2020), I also wanted to avoid any of the characters from sounding too menacing or scary for a child. I had a similar objective for the music selection. All the stock music in the prototype are instrumental pieces that were part of a family-friendly music library. The main theme song of the story includes original lyrics that I wrote with three characteristics in mind; simple, fun, and catchy. 

The sound effects evolved throughout production. I had originally planned for sound effects that would help support each scene such as ambient sounds of a room (eg. the sound of feet walking down a long hallway) and the characters’ movements and interactions with objects (eg. a character pulls a cork out of a bottle). Further research on VUI design prompted me to consider how sound effects could be used to provide the user a context on how to interact with the story (Pearl 62). For instance, a twinkling sound effect was later added to cue the user whenever it was time to make a choice in the branching narrative. This type of sound effect is considered an earcon because it is a piece of information that lets the user know that they are expected to respond to the system (material.io 2020). Another addition into the audio mix were various hero sounds, which indicated whenever the user had selected a choice that would advance the narrative closer to success. This could also be considered a form of feedback, which has been found to be effective in a VUI designed for children (Cieślak 2020).

Despite all the unexpected ways in which I had to shift production, overall I was pleased with the end result.  Once the produced audio clips were ready, I incorporated them into the conversational map that I created with Voiceflow. I was then able to test out the prototype with my children using our home smart speaker and observe how they interacted with the story. To witness their intuitive responses to the characters and get up to take part in some of the activities brought a huge smile to my face. As a creator, there is no better feeling than to see the content elicit a reaction from the audience, and on a new level, interact with the content. Observing my children playing with the interactive audio narrative allowed me to confirm what worked and didn’t work. The children’s interactions and reactions to the story confirmed that receiving feedback from the characters affirmed their role as the choice-makers. Music played its most important role as a cue for commands. The children knew that they could respond once the music had ended. Their attentiveness throughout the story was due in large part to their required involvement to determine the story progression (Miller 2014; Sperring and Strandvall 233).

The prototype test also revealed what needed to be improved. Using vocal commands to communicate with a smart speaker is intuitive but the moment the voice assistant cannot pick up a command, which happens more often with children, is when it becomes frustrating for a user. I also found it interesting that my children were more invested in chasing after the Messlins than to do the side activities. For them, that was the main objective of the game, and anything beyond that was an obstacle. In terms of the character design, I spent considerably more time on the development of the prince and princess characters because I had assumed a child would feel more connected to them because of their age and shared goal of finding the Messlins. So, it came as a surprise that my children were more engaged whenever a Messlin was in the scene. Although the Messlins had considerably fewer lines compared to the other characters, they were the most memorable characters because they were, as one of my kids described them, “silly and funny.”It is an exciting time for voice technology, and with huge brands like Disney and Hasbro starting to develop their own content for voice-enable devices it’s evident that we are just witnessing the beginning of a new way to connect with audiences. It is especially important these days, with everyone at home and relying more on screen time. Providing children with a new way to interact with content while also encouraging them to imagine the scene in their own minds is fulfilling to me as both a creator and a parent. In the coming months, I look forward to testing out the prototype with other users through the Ryerson University Transmedia Zone Incubator program of which I am currently a member. Further testing is an integral part of the process and I hope to gain valuable new insight on the user-experience.  Much like a branching narrative, where this creative process leads me to next is still unknown, but I’m looking forward to the adventure nonetheless. 

References

Buurman, H. A. Virtual Storytelling: Emotions for the narrator. MS thesis. University of Twente, 2007.

Calvert, Sandra L., and Barbara J. Wilson. The Handbook of Children, Media, and Development. Wiley-Blackwell, GB, 2008.

Cieślak, Katarzyna. “How to design a Voice-First game for kids? – Voice Tech Global Medium.” Medium, 24 Jan. 2020, medium.com/voice-tech-global/how-to-design-a-voice first-game-for-kids-79ec345d0c35.

Crawford, Chris. Chris Crawford on interactive storytelling. New Riders, 2012.

Hall, Erika, and John Maeda. “Conversational design.” (2018).

LudoNarraCon. “Developing Branching Narratives.” YouTube, uploaded by Fellow Traveller, 5 May 2020, https://www.youtube.com/watch?v=mgD81pQlu1o.

Material.io. “Applying sound to UI.” 20 March 2020, https://material.io/design/sound/applying-sound-to-ui.html 

Markopoulos, Panos, et al. Evaluating children’s interactive products: principles and practices for interaction designers. Elsevier, 2008.

Miller, Carolyn H. Digital Storytelling: A Creator’s Guide to Interactive Entertainment. Focal Press/Elsevier, US. 2008; 2014;.

Pearl, Cathy. Designing voice user interfaces: principles of conversational experiences. ” O’Reilly Media, Inc.”, 2016.

Phillips, Louise. “Storytelling: The seeds of children’s creativity.” Australasian Journal of Early Childhood 25.3 (2000): 1-5.

Ryokai, Kimiko, and Justine Cassell. “Computer support for children’s collaborative fantasy play and storytelling.” CSCL. 1999.

Sperring, Susanne, and Tommy Strandvall. “Viewers’ Experiences of a TV Quiz show with Integrated Interactivity.” International Journal of Human-Computer Interaction: Social Interactive Television, vol. 24, no. 2, 2008, pp. 214-235.

“The evolution of video games as a storytelling medium, and the role of narrative in modern games.” 6 Apr. 2020, www.gamasutra.com/blogs/ChrisStone/20190107/333798/The_evolution_of_video_games_as_a_storytelling_medium_and_the_role_of_narrative_in_modern_games.php.

Treasure, Julian. “The 4 ways sound affects us.” 13 Mar. 2020,  www.ted.com/talks/julian_treasure_the_4_ways_sound_affects_us?language=en

Vygotsky, Lev S. “Play and its role in the mental development of the child.” Soviet psychology 5.3 (1967): 6-18.

In introducing my new novel, author Ryan O’Neill puts it most succinctly:

This is an introduction to a novel you will never read.

He adds hastily that he is referring not to the book in your hands, the one he hopes you’re about to begin, but the novel that inspired his words, the novel he read.

The novel in question is Ex Libris and regardless of which copy you read it contains twelve chapters that can be shuffled into any order. The number of variations possible with such a structure is a little over 479 million. It has been published in both standard paperback and ebook editions, each copy a newly shuffled order of chapters unique to that copy alone. The manuscript that Ryan read in order to create his introduction is different to the finished copy now in his possession, which is in turn different from every other copy ever made.

I have written about Ex Libris previously where I noted that this kind of storytelling has its precedents, the most significant of which all hail from the 1960s. Nanni Balestrini’s Tristano was conceived and written using early computer programming to randomise its content between copies, though it wasn’t published as intended until print technology had caught up in 2007. Other similar books were housed in a box, either as loose leaves (Composition No. 1 by Marc Saporta) or as chapter booklets (The Unfortunates by B. S. Johnson). Of these, Johnson’s novel provided the most direct influence on the structure of Ex Libris: the fluid pieces of the story are defined not arbitrarily by the size of the page, but by the narrative itself. The story is broken into discrete, meaningful components that combine to form a larger picture.

What Ryan alludes to in his opening statement is that any work structured in this way presents a challenge to critical reading. How can readers universalise their experience if the texts they read are never consistent? You may disagree with someone else’s reading of a text, but you do so on the fundamental understanding that both of you have at least read the same words in the same order. John Bryant’s scholarship on textual fluidity through editions, translations, and adaptations demonstrates that texts are never as concrete as we might assume. But variation between editions is a long way from a narrative that changes by design between individual copies. Although it is possible to arrange Ex Libris in approximate chronological order (some events in the story clearly happen before others), each of the novel’s fluid chapters is a vignette, dependent on the others for context, but not for prior knowledge. I have used the analogy of a jigsaw puzzle to explain this to readers: smaller narratives link together to form a larger picture. The order in which the pieces are placed changes the individual’s progress but doesn’t change the ultimate picture.

The coding to compile finished print-ready files is done in Automator, the computer equivalent of a Rube Goldberg machine.

It can be difficult to get past the structure itself and the mathematics behind it as many contemporary and more recent reviews of recombinant works demonstrate. Umberto Eco in his introduction to Tristano, focuses almost exclusively on the novel’s number of permutations with only a cursory nod to the story. This might be understandable for a novel that, though beautiful, has a deliberately tenuous grip on character, plot, and setting. But the same approach is repeated in reviews of Saporta, Johnson, and other similar works. It is as though the flashy acrobatics of the novel’s physical construction obscure what the writers are doing within. And the critics’ resulting performative bewilderment or pithy dismissal of a wacky experiment seem to me like missed opportunities.

When the assumed shared experience of an audience is modified or removed altogether, how does their engagement with a narrative change? Some clues may be found in my own experience on both sides of the reader/writer divide. How I initially read and thought about a fluid novel like The Unfortunates, for example, is very different to how I have come to think about Ex Libris and that change in point of view has been illuminating.

My experience with The Unfortunates suggests that a first reading looms large in one’s perception of story. While reading, I had to keep reminding myself that the clever positioning of two adjacent scenes was attributable not only to the author’s craft but also to sheer happenstance. We’re trained to read stories as linear and it’s a hard habit to break. When I return to The Unfortunates today, no matter how many times I reshuffle its contents, the story is always coloured by that first reading and how the chapters initially unfolded. That first reading has become my definitive version of the novel from which all others deviate.

Readers of Ex Libris may have a similar experience, perhaps moreso given their copy cannot be physically reconstructed. Information that colours the perception of the characters and their actions may come earlier or later and its impact will undoubtedly shift. Readers who see more of a particular character earlier, for example, may centre the story around them in a way others won’t. Several of the fluid chapters also contain crucial pieces of information that change a character’s image or motivation and cast events elsewhere in the story in a different light. Reviewing the chapter order for each copy, I frequently pay attention to where these chapters fall, wondering how their precise location changes the tenor of the story.

I say I wonder because, primarily, I must rely on guesswork. My perception of the novel is not of a puzzle but of narrative pieces in constant motion, a true fluid state. As I worked on it, Ex Libris formed a kind of web, a set of interlocking shorter narratives that fed into a larger complex. For me there can never be a definitive version of the story, only discrete narrative chunks that cross-reference, echo, or contrast, but never line up precisely.

Unlike The Unfortunates which can be endlessly reshuffled, Ex Libris is presented to the reader as a single, fixed manifestation of the narrative. But it’s also a window, a viewport through which you might catch a glimpse of what I see. Without the capacity to physically manipulate pages, the reader must instead imagine that fluid state and the differences in emphasis that come with changes in how the story unfolds. With Ex Libris, like with all fluid texts, a critical reading should regard not only the text as it’s presented, but also with the text in every conceivable other version. The success or otherwise of any one version of the narrative is merely a subset of nearly half a billion possible narratives in the aggregate. Though difficult to fully conceive, this is something I suspect many readers instinctively know. A common reaction from those who have finished the novel is to seek out other readers to compare notes.

But what readers who squint to catch glimpses of the author’s view may not realise is that they have experienced the story in a way I cannot. I can cast an eye over any number of versions of my story, but I can never see the flow of a linear narrative, only a single path running through that fluid web of chapters. For better or for worse I can never have the experience I had reading The Unfortunates.

I suspect that’s why the story that emerged turned out far more self-reflexive than I had originally intended. Maybe it was inevitable that a narrative featuring a band of literary misfits reconstructing a library from fragments in a dystopian world would eventually turn in on itself, a comment on how fiction can become a vehicle for revealing how we construct our own truths. In the same way the story’s characters can never truly reach the author, so too a reader’s and writer’s experiences always remain tantalisingly out of reach for each other.

Ex Libris is out now.

Reference

Bryant, J., 2005. The Fluid Text. Ann Arbor: University of Michigan Press.

Like all of us, March 2020 was a slow decline into horror at the news of Covid-19. While the British government dithered about what to do, I cancelled some events I’d been booked to hold in promotion of my novel The Blame Game. Quite frankly, I was afraid to travel (from Glasgow to various parts of England), given the news reports about the virus, but I was disappointed to have to scupper these long-held plans – what a pity, I thought, that I couldn’t organise a virtual event in place of a physical one.

 On 13th March I tweeted something to this effect – it seemed likely that locking down the country was going to be necessary (though the British government would dither for another 10 days about it) and I posted that it would be a good idea to have a literature festival that was entirely online. The tweet had a big response, so big, in fact, that I felt I’d need to put my money, or at least my expertise, where my mouth was. Having convened the Distance Learning MLitt in Creative Writing at the University of Glasgow since 2015, I have experience in running virtual events, and in using the technology. In short, I knew both the pros and the cons of virtual events pretty intimately, and so I felt relatively confident about doing some online author events for people who were stuck at home to access. Paper Nations, a well-known Creative Writing incubator, got in touch with me and asked to partner on the festival, which I decided to call the Stay-At-Home! Literary Festival. I created a google doc and cascaded it to writers who’d expressed interest, to which they added their proposed event with a date and time. I was astounded at how quickly and enthusiastically people responded to my call. In addition to over 100 writers quickly filling up the google doc (forcing me to extend the length of the festival to 16 days), an organisation called the Professional Writing Academy got in touch, offering a range of fabulous writing workshops for free.

 Within 11 days, the Google doc had exploded with workshops, talks, open-mic nights, panel discussions, and showcase events. Word spread quickly to publicists, who were eager to find new routes by which to publicise new books and replace cancelled events. I had to scramble to find slots for authors; I knew all too well how disappointing it was to have to cancel a tour. I also wanted to ensure that minority voices were heard; I love attending book festivals, but all too often I have felt frustrated by their lack of diversity. I’m also deeply aware of how festivals can exclude people with caring duties, with disabilities, and with work commitments. I’m very aware of people who feel out of place at literary festivals, and who simply can’t afford to attend.

The Stay-At-Home! festival was programmed on zero budget. I made it clear to everyone who expressed interest that, while I believe fervently writers should always be paid for their work, this was a pro bono festival; everyone involved, with the exception of 7 Writers in Residence who were paid a small honorarium by Paper Nations, contributed for free. By programming this festival, I was able to platform a diverse range of authors. I wanted to ensure that minority writers were heard, and that writers at all stages of their careers could be involved. One of my MLitt students at the University of Glasgow offered to host an open-mic session; the available slots filled up within hours. Similarly, writing workshops attracted crowds of 500+ – an outstanding feat, and an impossible one in physical form. I was witnessing the birth of something truly unprecedented – the potential of virtual live literature in all its awkward, brilliant glory.

There were many learning curves. One was the phenomenon of Zoom-bombing. Naively, I thought that sharing the Zoom link to an event on social media was danger-free. As it happens, there are individuals out there who get their kicks from troll-bombing such events with the intent to either shout racist slurs across the microphone, or use the screen-share facility to splash porn across the screen of unsuspecting attendees. This is horrifying to experience, and I learned fast how to stamp it out. But even when I had a handle on muting attendees and preventing screen-sharing, trolls would use the chat box and, on one sickening occasion, their own video screen to share inappropriate content.  

It is saying something, however, that feedback from our attendees – of which there were 14,689 from all over the world – rarely mentioned the events that had been subjected to trolling. Instead, feedback focused on the sense of community that had been created by the festival, and it is this which I’m most proud of. Something was created on 27th March 2020, at the very beginning of lockdown and when we were all staring into the surreality of a global pandemic, that has triggered a host of other virtual events. Whereas Edinburgh Book Festival initially announced that it was cancelling the 2020 festival, it has done a U-turn on this decision, with a highly successful virtual festival in its place. Several of the authors who led events at the Stay-At-Home! Festival went on to run their own festivals – for example, Dr Pragya Agarwal (author of Sway) led a festival on South Asian Writing, Virginia Moffatt ran a New Authors festival, and I was pleased to consult on a number of others. 

The Stay-At-Home! Literary Festival was a wonderful experiment. I had the rare privilege of bringing together a range of authors and audiences from all over the world to talk about literature in a moment when we were all facing the terrifying prospect of quarantine and a deadly virus stalking our streets. Lockdown also permitted me a rare privilege of experimenting without too much fear of failure – as one of the very first digital festivals, I was venturing out on to new territory. We were all confined indoors, and therefore my audiences engaged with authors – and each other – from their interior spaces. And yet, I found a richness in encountering high profile authors in their office, kitchen, or spare bedroom. Stay-At-Home! Festival facilitated a levelled audience experience. During the project I was keen to encourage authors to embrace the intimacy, immediacy and even the informality fostered by an online platform like Zoom, and I believe this worked well to draw down certain barriers. Going forward, I don’t think we should be too quick to eradicate this from virtual events. It’s one of the many strengths of virtual live literature.

I have no doubt that the landscape of live literature has forever been changed by Covid, and possibly for the better. I would bristle slightly when I’d hear people say ‘unfortunately we have to hold this event online’ – bristle, because for many people, online was the only way by which they could be included. During the feedback plenaries that I held after the festival, many of the SAH festival’s attendees commented that they found the accessibility of the festival ‘gold standard’. Some posted pictures of themselves with babies on their laps while they engaged in events. If we continue to put thought and experimentation behind our efforts to make literature more inclusive, virtual live literature can boldly take us places that we never knew possible.

Matt Finch writes and helps communities, companies, and institutions around the world to do useful and surprising new things. His latest digital work is the interactive narrative, The Library of Last Resort.

You have a varied background including a Ph.D. in Modern Intellectual History. How did you arrive at this nexus of strategy, storytelling and technology?

I wrote a Ph.D. about people who fled the rise of fascism in 1930s Europe and how they adjusted to live in their host countries, including the stories they told about their pasts. At the same time I did work with asylum-seeking children and then a stint as a kindergarten teacher in England. I also wrote travel guides, magazine articles, and worked in local government and the tech sector. Increasingly, people asked me to work with them on high-level strategy, or getting a wider community involved in conversations they were having. 

I suppose all of those jobs were to do with relationships, and questions, and the stories we tell ourselves about our lives: where we’ve come from, who we are now, where we’re going next. 

For potential clients you describe your work as “scenario planning and foresight, policy consultation and strategic direction, plus facilitation and professional development”. How would you describe it for a broader audience, or for people who might take part in one of your sessions?

I help people, communities, and organisations to make better decisions about what they want to do in the future. Sometimes that involves imagining the futures which might await, in order to expand our understanding of what’s going on in the present. That’s what people call foresight, as opposed to forecasting, which is the traditional notion of trying to correctly predict the one future which will definitely occur.

Most recently, I’ve worked with Energy Consumers Australia to imagine the energy sector of 2050 and with the University of Oslo exploring the future of digital technology in schools. I’m currently advising a project called IMAJINE on the future of regional inequality across the European Union. It’s fun and rewarding to help people explore their strategic blindspots.

What are the creative works that have most inspired you?

I could and probably should reel off a whole bunch of writers and artists who have stayed with me and who I want to be associated with, but really I think that everything you take in inspires you. Right now I’m absorbing a bunch of Gail Simone’s glorious comics; José Esteban Muñoz’s Cruising Utopia, about queer identity and the future; and the catalogue from an exhibition of works by the surrealist Dora Maar. All of those are massively feeding my head.

The story of inspiration I most want to tell comes from my kindy teaching days. One afternoon, out of the blue, this kid Josh said, “I love melon. My mum says if I eat too much melon, I might turn into one. I could become a superhero…Melon Boy!”

He started laughing, absolutely killing himself with laughter, crying, doubled up, the whole thing. I think it was the first time he had ever made himself laugh in his whole life; he was almost surprised at the reaction he’d triggered in himself.

It was so cool. We stopped what we were doing and ended up making a Melon Boy comic together as a class, piecing together the story one image at a time. (A malevolent witch tricked Melon Boy into losing his powers by feeding him so much cake he became Cake Boy). Everyone had so much fun and was so into it; and it all came from this first moment of Josh surprising himself. I find those moments, those sparks, inspiring.

You work a lot with libraries, notably as creative in residence at the State Library of Queensland and Creative/Researcher at British Library Labs. What is it about libraries that has made them particularly receptive to your work?

In the information age, it’s fascinating to see libraries change with the times. Libraries are about discovery, not instruction; it’s a different power dynamic to other knowledge institutions, more open-ended and exploratory. There are also some significant tensions as our notions of the public and private shift. But even in the shelfiest old library of the past, the user went in, chose a book for themselves, opened it, made meaning for themselves as they read. That’s what I hope we can take with us into the future from the library tradition.

A library should be a place where communities connect with knowledge, information, and culture on their own terms, and that could even mean a place where the professional gatekeepers abdicate their power or are radicalized, letting themselves be surprised and led by the community they serve. There are things to learn from Ludi Price’s work on fanfiction archives, Audrey Huggett’s immersive play experiences in libraries, and grassroots “fugitive libraries”.

From your observations around the world, do you see trends or outliers today that may point the way for libraries to thrive into the future?

The world is changing so much and so fast, it’s difficult to make predictions. I also don’t think that you can necessarily copy-and-paste what works in one context to somewhere else. Good strategy is about making a diagnosis specific to your circumstances and then taking a smart bet on what you ought to do next. I think that great libraries now and in the future will be deeply attentive to the current and emerging needs of the communities they serve and which fund them.

How are libraries adapting to an environment where staying at home and social distancing are essential for the public good? Do you see these adaptations remaining in place beyond the pandemic?

Martin Kristoffer Bråthen, a Norwegian librarian, has written and spoken about this, asking, in an age of lockdown, “What is the library’s value if they focus on being the middleman between digital content and an online consumer?” 

I suspect that the changes libraries are making to adapt to the pandemic will be like those being made in wider society; some of them will stick because they are more desirable or more efficient. When there was a strike on the London Underground a few years back, researchers tracked the journeys made by commuters when their usual journey to work became impossible. A significant number of travellers stuck with their alternate routes after the strike ended; the crisis had actually shown them a more efficient way to get from their home to work and back each day.

In the long run, while some changes will stay, others could revert, and yet others will shift into even more novel and unfamiliar configurations. It’s nice to imagine life “beyond the pandemic” but I suspect we have a sustained season of turbulence ahead of us, not just COVID-19 but all the other social, economic, environmental changes which might now shake up our way of life.

Your most recent creative work is an old-school branching narrative, set—of course—in a library. Why did you choose a branching narrative design for this particular story?

I think a lot about the balance of power between author and audience. We talk about interactivity, but mostly it’s just inviting people to make choices from a set that has already been devised for them. Library of Last Resort was an experiment in finding the limits of that framework, and then trying to jump beyond those limits to a place where the person who starts as the reader can do something which the author couldn’t see coming, enlisting them as a creator and someone who can surprise others, forcing them to confront the blank page.

I’d previously written a “choose-your-own book review” in a similar vein for an Aussie arts journal, and through them I met the excellent and assiduous editor Adalya Nash Hussein, who worked with me on the Library of Last Resort. Her insights improved the text and structure, making the Library a better, richer place to visit.

The Library of Last Resort occupies that very blurred space between “game” and “narrative”. Do you lean towards one or the other label when framing the piece? Are such labels even helpful?

I like a good blur! 

If you approached it as a story, it’s probably quite frustrating because there’s a lot of wandering around and extraneous material in there – I wanted people to have the sense of getting lost in a collection, overstuffed with reading, before they made their escape. I think that happened to you when you first entered the Library, Simon – you had to ask me if there was a point to it all, or the point was just to get lost!

If you approach it as a game it’s probably equally frustrating because there’s only a token sense of mission or victory! I’m not really into keeping score. There is a hidden ending where you can escape from the Library in a hot air balloon; one of my playtesters found it on his first playthrough, just by making the choices that he would make if he was really in the Library. Some people’s brains are just wired that way, I guess.

Maybe the Library of Last Resort is an experiment in frustration and release…I think one of the hard things about trying something new is figuring out how to work with people’s expectations. When you click that link, do you want to be told a good story? Do you want to be given a good puzzle, with the satisfaction of finding the “right” solution? How much effort should you be expected to put in? How much uncertainty should you experience?

You present The Library of Last Resort as a form of escapism, but the story contemplates fundamental ideas around the nature of play and narrative, as well as truth and objective reality. How important is it for you to strike a balance between having fun and addressing some of the deeper complications of contemporary life?

When you’re a kid, the world is so new to you that you’re constantly exploring surfaces and probing the depths, asking the big questions, where do we come from, why does this happen. It’s also an emotional journey: losing your teddy bear can feel like cosmic despair, but jokes about eating too much melon can conjure sheer delight. All of that – the deep stuff, the superficial, and the make-believe – mixes with the everyday and apparently trivial. That’s a cool place to be.

I’m not pretending the Library of Last Resort gets anywhere near what Josh achieved with “Melon Boy”, but it’s nice to have something to aim for.

Find out more about Matt at mechanicaldolphin.com.

Mez Breeze is one of the leading innovators in the fields of experimental storytelling and virtual reality literature. Her work has heralded new ways of making and collaborating for creators, and her ethos and politics demonstrate ethical practices for all creative practitioners.  In 2019 Mez was awarded the Marjorie C. Luesebrink Career Achievement Award to honour her decades of experimental work in the field of electronic literature. The Writing Platform is thrilled to celebrate this accolade with an interview about her practice, influences and identity and the cryptic nature of her virtual global presence.

Your work straddles a number of forms and fields, do you consider yourself primarily a writer? Or a storyteller or digital artist? Or all of these things?

My work does absolutely stretch across [and through, and in-between] a diverse range of fields and forms, with my practice continuing to be a type of massive sandbox experiment. In relation to whether I consider myself primarily a writer, I tend to [mostly] think of myself as a “creative” [even with the awful hipsterish/advert-like baggage that the term “creative” comes with nowadays], like a type of digital tinkerer. At the moment I’m basically a combination of digital artist/writer/storyteller, electronic literature creator, games developer/designer, VR sculptor, and Mezangelle-crafter, so it’s difficult to pin down an exact term that encompasses all I do, and how I do it: mostly because of the need to constantly explore + experiment + play + learn + problem solve [ie curiosity is my bff].

It’s been an interesting past two, three years in relation to what I’ve been creating with Spatial Computing tech [Virtual Reality and more broadly XR (Extended Reality)] and how this impacts how I view, and label, what I do. In 2017 + 2018, after coming to the difficult realisation that I needed to examine how [and with whom] I was making funded creative work[s], I began seriously shifting my creative focus back to skills related to the plastic arts, skills that I hadn’t properly used for over 20 years. This was also an attempt to swivel away from what I’d become known for creating/writing, choosing instead to pivot towards the unknown [something I have a history of doing when creatively constrained or pigeonholed]. It was at this time too – after getting up the courage to prioritise professional standards and behaviours [especially regarding one digitally-born series I was involved with, in regards to behaviour considered acceptable by other leads of the team] – I moved away from a more traditional business-centric perspective and shifted back to creativity for creativity’s sake by forging connections with a group of incredibly motivated, generous, and positive XR Artists. Encountering such a diverse, considerate and talented crew was like a breath of fresh air, and meeting them has been such a motivator to continue crafting 3D and VR models that I’ve since incorporated into my digital writing projects. To put this in context, I’m not officially a sculptor or 3D artist [though I was trained in the visual arts in the early 1990’s], so launching into this area was a tad daunting, but I’m loving the challenge, the sense of community [and common decency], the comradery, the constant learning processes involved, and how I’m able to fold all this into new digital literature and storytelling. 

What is your approach to technology in relation to story? Is your aim a coming together of form and content? Is technology a creative tool for storytelling or the foundation for the story?

The technology I employ when creating stories I do consider [first and foremost] a tool. It’s a method of forging [and/or serendipitously breaking, playing, prodding, testing, reformulating, poking, and reforging] creative output [both form and content]. In terms of using tech to tell [or form] a story, my aims are [and this is where experimentation + play + learning raise their heads] dependent on my overall goals. If the goals are to create a story with a definitive structure and outcome [say where there’s a team involved, or if a project has commercial intent], then my aims are *very* different to a project where I can be more expressively elastic. There’s a fine line between producing open-ended and freeform tech experiments with no critical assessment and experimentation for its own sake coupled with being rigorous about the quality of the final output. It’s a fine balance, and it’s often hard to achieve.

Are there affordances in technology that allow you to write/create in ways that are more impactful for audiences?

There are certain aspects of technology that I’ve enjoyed molding and contorting into various states over the years, and a byproduct of this is that [fortunately] audiences do seem to resonate with the results. Often I’ll take a punt on various technologies [like the Vive Pro VR rig, Vive trackers or haptic feedback gloves] that I hope will shape my creativity in interesting [and unexpected] ways, which I’ll then assimilate into projects that hopefully make sense to an audience. I became interested in the idea of using technology to craft works that an audience would find impactful when, in 1994 [or 1993?] I was first exposed to the work of the Australian Cyberfeminist Collective VNS Matrix. Their mix of feminism, text/image merging and virtual engagement intrigued me, as at the time I was creating mixed-media installations involving painting, computer text and computer hardware. I first dove into the Internet in 1994 using Telnet/Unix to explore avatar use, identity-play and interactive fiction: it was here that ease-of-dispersal and temporality were incredibly important for both the production and the absorption of such works [often the audience were the creators, and vice versa]. Mezangelle had its roots here, initially evolving from immersion in email exchanges, computer programming languages and chat-oriented software [ie y-talk, webchat, and IRC]. 

Mezangelle is highly conceptual and fuses traditional language conventions, image and text, programming code, social commentary, and online communique´ – it’s been intriguing [and occasionally irritating] niche audiences now for over two decades. I’ve gradually been integrating Mezangelle into more expansive projects for a while now, including complex narrative game environments and projects like my 2018/2019 Virtual Reality Microstory Series V[R]ignettes.

What projects have inspired you?

Are natural ecosystems a project? If yes [or even if no], I draw intense and constant inspiration from them, from closely observing all types of wildlife [crimson + eastern rosellas! blue-tongued lizards!] including pollinators [blue banded bees! caper white butterflies!] in the permaculture setup I have here on Gundungurra land, to microclimate-crafting – I get such frissonic ideas when fostering [and reflecting within] these environments. Also, inspiration strikes in the most ephemeral of places, and for the most part what inspires me aren’t creative works as such, but if pushed to picked some inspirational projects they’d be: Surge by Arjan van Meerten, Firebird – La Péri by Innerspace, Patchwork Girl by Shelley Jackson, World of Warcraft by Blizzard Entertainment, Ænima by TOOL, 1984 by George Orwell, The Cells by Louise Bourgeois,  The Rime of the Ancient Mariner by Samuel Taylor Coleridge, and The Endless Forest by Tale of Tales.

What technologies and/or platforms are you excited about currently, and what do you see coming down the line?

I’m tingle-excited about accessible, responsible and affordable spatial storytelling tech and associated platforms, and am currently using such tech to create works like the collaborative XR Story Series V[R]erses. What I’m less excited about is the use of XR tech to promote project/platform siloing [think: privacy concerns related to Facebook-owned Oculus products and the many questions over the retention of Oculus generated data, transparency regarding how this data is used – including sensitive metrics to do with body/haptic monitoring and voice recording – and who they allow to access it]. This concern part-explains my shift in 2017/2018 to using XR platforms in a different way, with projects like V[R]erses, A Place Called Ormalcy and V[R]ignettes created specifically to be experienced variably across a wide range of devices, computers, and platforms. Unfortunately, a lesson the death of Flash has taught us is that VR may unfortunately be headed along this same route, where certain hardware and distribution channels become more and more restricted and siloed [think: Google recently binning the VR Daydream headset and switching its new gen Pixel phones to AR, Samsung pulling the plug on GearVR, and Facebook producing a range of tetherless headsets that could lock users into a Terms-of-Service-loaded surveillance nightmare]. In 2017 I saw this shift coming and pivoted to make sure at least some of the VR-centric projects I was producing had access concerns right at the forefront [as well as only using VR hardware and software that didn’t/doesn’t perpetuate this – goodbye Oculus headsets and apps!]. As a result, I decided to try to create works that audiences can experience in WebVR [browser-based] environments that are openly optimised for mobile devices, desktop computers, and VR headsets. It’s incredibly important to me that tech such as AR and VR doesn’t just become a vehicle for the privileged, where only those who can afford to experience such walled-off, high-end works [and indeed, produce such projects] become the standard or default. After coproducing high-end works for quite a few years, it really did open my eyes to how such projects can perpetuate inequality and exclusivity – the opposite of what I want my work to convey.

What I see coming down the line can be summed up by this quote I gave earlier this year [with a liberal dollop of AI and neural networks/ DeepFakery utilisations thrown in]: “I suspect that VR as an industry definition is probably going to be subsumed into the category of XR [Extended Reality] or SR or SC [Spatial Reality/Spatial Computing] as those seem to be the broader industry terms scrabbling for purchase at the moment. From my POV it’s been really interesting to see another tech hype curve go into what I term “Peak and Trough Syndrome”, just as has happened with AR – you have industry types predicting that VR and AR will be the next big thing, only to have actual functional constraints [such as comfortability, tetherless gear making a less-than-definitive-dent in the overall market, lack of robust blockbuster VR content production and audience uptake, platform wars/fragmentation, monetisation dilemmas, etc] all affecting mainstream adoption rates. I am keenly following how VR social spaces and VR arcades will fare over the next few years, as well as the swerve away from VR/AR/MR as the next massive tech trend to such tech being viewed and conceptualised as an integrative social service and/or “virtual beings” vehicle [especially when combined with AI, like with 3Lateral’s acquisition by Epic, Magic Leap’s Mica or VR Story Studio Fable’s relaunch with digital human AI/VR agents as their focus].”

Your work has a philosophical and political dimension: do artists have a responsibility to respond to the current state of the world?

Abso-fecking-lutely! Artists shouldn’t just have a responsibility to creatively respond to the current state of the world, they should also have a responsibility [as does everyone] to attempt to create beneficial change. From my own section of the globe, I’m constantly trying to effect relevant social critique/change with what I create [and how I create it]. Two recent[ish] projects from the past few years in which I’ve tried to convey a sense of this social commentary/responsibility are the Virtual Reality YA Adventure Perpetual Nomads, and the dystopian VR story A Place Called Ormalcy. Perpetual Nomads [or PN] is an Australian-Canadian Coproduction that’s part of the long-running Inanimate Alice digital story franchise. In PN, a VR Experience, you play as Alice where [SPOILER ALERT] you navigate creepy scenarios like finding yourself on the pointy end of a harassment stick, engaging in social app-based tug-o-wars, and attempting to cope with your phone battery running crazily low just when you really need it. But underlying all of these everyday challenges lies a more sinister one – you’ll have to play it to find out what it is, but let’s just say as writer, narrative designer and creative director of the project I wanted to ensure the message it conveys concerning privacy, surveillance, and greenwashed corporate corruption isn’t designed to be subtle [a hint: if you do dive into this work, do make sure to play PN right through to the end, past the credits].

Like PN, A Place Called Ormalcy is a work that also has social commentary at its centre. It has multiple ways of being experienced, read, and viewed – the audience can choose to experience it as a simple text-based dystopian story [with a profound warning for western democractic functioning], as a 3D set of tableaus with text annotations, or in a full-blown VR arena where they are able to teleport around within each chapter. With works like PN and A Place Called Ormalcy, there’s massive potential to get under an audience’s skin, and to construct experiences that challenge prevailing orthodoxies. What I tried to do with A Place Called Ormalcy in particular was slip under a user’s radar by offering up a child-like allegory that could be read/experienced as simplistic, but that also has deadly serious social commentary at its core.

You have an international reputation despite the fact that you do not travel for work and usually appear at conferences via Skype or other remote meeting platforms. Can you tell us a bit about how you manage this aspect of your life?

In the early 1990s, I was exposed to two life changing infonuggets. The first was learning about VNS Matrix [as I talked about above], and the second was learning about [and starting to implement] the sustainability system “permanent [agri]culture”, or permaculture as it’s more commonly known, alongside discovering eco-pioneers like David Suzuki and Bill Mollison. Encountering VNS Matrix then led me to collaboratively producing interactive fictions + games + experimental output with people has diverse as University of Melbourne students to Palo Alto engineers to Swedish computer geeks in the 90’s, and it taught me that identity could be used in this [then] new frontier-like space to subvert and collapse associated prevailing biases to do with gender, race, and age. This then linked to the realisation that [back then] such digital/virtual spaces could help alleviate certain prejudices + biases associated with gender labelling in patriarchally-infused fields. To some extent this stuck, and fed into the formulation of a growing plan to reduce my everyday effect on the burgeoning climate emergency [yup, it was even a thing even back in the 1990’s, though the phrase “climate change” wasn’t nearly as buzzworthy then as it is today] by deciding to devote myself to creating and maintaining working permaculture systems [and monitor, while actively reducing, my carbon impact] where caring for the environment was/is of utmost concern. I live [with] this concern on a daily basis, and have accordingly molded my everyday life around it. This decision was made in tandem with continuing to dismantle traditional constraints placed on women and minorities [in creative and technological spheres] through remote participation. The fact that embodying a digital, rather than physical, presence could help solve various issues like these absolutely influenced, and continues to influence, how I manage all aspects of my life. There’s also another secret-squirrel reason why I mostly choose digital participation at conferences and events, but this is part of a special longitudinal project that will be revealed in good time – some select few know about this, but that’s where the knowledge will stay just for now [and heh, could I *be* more cryptic?].

Lastly, what are the advantages and disadvantages of being a working artist located in Australia?

Advantages: the isolation. Disadvantages: the isolation.

Website links:

Main Site: mezbreezedesign.com

Patreon Site: patreon.com/mezbreeze

Twitter: twitter.com/MezBreezeDesign

When writers discuss plot and pacing in narrative craft, especially in creative writing classes, we often talk about the curve of stories, the rise and fall in tension that characterises the most common story structures. Now usually, at least in my experience, that curve is not something a writer actively thinks about while composing a work. It’s more instinctive. Manipulating pace is one of the writer’s primary tricks in taking a simple sequence of events and turning them into narrative. But what in retrospect looks deliberate and disciplined, is in the act of writing more like manipulating the feel of the story as you go.

When it came to my current publishing project, all that instinct counted for nothing. An experiment in recombinant narrative structure requires careful consideration and active manipulation of the curve.

Ex Libris is a novel containing twelve chapters that can be shuffled in any order, yet always presents as a cohesive narrative arc. It is being published in a print run that randomises the chapters between each copy. With close to half a billion possible combinations, each copy will contain a unique version of the text, yet all will tell the same story.

The title for ‘Ex Libris’ comes from the nineteenth century fad for bookplates.

The two books that, more than any others, inspired the structure of Ex Libris are The Unfortunates by B. S. Johnson and Tristano by Nanni Balestrini. Curiously, both were written in the 1960s, though Tristano wouldn’t find its true form until 2007.

The Unfortunates is a beautiful but restless story about grief and the intrusion of memories that overlay the banality of daily life. The novel was structured with a fixed opening and closing and with freely fluid chapters between. The first edition and its more recent reproduction was published as chapter-length booklets contained in a box, which the reader was free to arrange in whatever order they desired.

Balestrini envisaged Tristano as a standard bound work with content that was randomised between copies. Sound familiar? The author was unable to realise the work as intended until forty years after its initial publication and with the advent of digital-based print technology. As the title suggests, Tristano builds its text using Tristan and Isolde as scaffold, which frees Balestrini to desiccate the narrative into the smallest of fragments, hints of meaning that only ever briefly come into focus.

Both works experiment boldly, not just with structure, but also with the language itself. The result is intoxicating: as a reader you feel like you’re having fun, even as you stumble around the text, constantly trying to find your footing. Tristano is one of the best examples of what I call ‘narrative drift’, the sense that, as a reader, you must let go of any sense of structure or meaning and allow the pages to take you wherever they lead. The Unfortunates is more focused, a narrative that initially drifts, but tightens as more of its pieces fall into place.

When I began writing what would become Ex Libris, I didn’t have a particular structure or publishing method in mind. What I wanted to do was write a work with fluid text without sacrificing a reader’s sense of plot or narrative arc.

I started with much more complicated mechanics and elaborate concoctions of fixed and fluid chapters. I ground my way through three drafts of the story, never completely satisfied, trying to find some magic key that would unlock how the story should work.

Eventually, I abandoned these versions of the story altogether. After a break from the manuscript, I returned and found myself back at first principles. Finally, I contemplated the curve.

I created a storyboard of sorts in Scriviner—movable lists in dot points—obstinately refusing to write anything resembling finished prose until a supporting structure had been mapped in sufficient detail. Slowly, a new structure began to take shape. The story begins in media res, at the beginning of the climax. Then it backtracks. It fills in details and circumstances that led directly to the opening scene. Then it jumps to the rest of the climax and conclusion. This means Ex Libris, like Johnson’s The Unfortunates, opens and closes with fixed chapters that frame the narrative. I had hoped not to invite such direct comparisons with Johnson, since clearly I would come off a distant second best. But the structure he pioneered, with its parallels to classic storytelling technique, is compelling in its simplicity.

Beyond the framing device, the fluid or recombinant chapters in Ex Libris primarily concern themselves with exploring character and world. These chapters exist in a weird state of semi-independence. A fluid chapter is episodic, with its own miniature arc. It cannot rely on prior knowledge. That doesn’t make it a short story. Although it shares traits with the short story form, a fluid chapter’s raison d’etre is to contribute to a greater whole. Detached from their surroundings and the framing of the novel, these little stories might struggle to pass a ‘so what?’ test.

Story and the structure developed in tandem. Part dystopia, part satire, with doses of paranoia and farce, and a self-reflexive bent, the novel is set in a hyper-networked surveillance state that has abandoned and almost forgotten the book. It focuses on a small band of subversives who collect the fragments and scraps of stories left behind. Calling themselves the ‘free readers’, they are attempting to rebuild a grand library they know must have once existed. A fragmented book about fragmented books, Ex Libris both feeds off and contributes to its own structure, a virtuous cycle of knowing winks.

I was very conscious of the reader’s experience.

I was very conscious of the reader’s experience, signposting and orienting the text at every opportunity to counter and minimise the sense of narrative drift. I maintained strict upper and lower word limits for each chapter. Too long indicated waffle that needed to be broken up. Too short pointed to a lack of substance. Often throughout the long planning stage of the project, I would stare at a dot-point breakdown for a chapter and think ‘but where’s the story?’.

I also avoided working on chapters in any particular order. Instead, I jumped around. From its initial use as a storyboard, Scrivener became a kind of reference tool as I wrote, a way to maintain a wide-angle view of the story, while moving the chapters around. The texts themselves were composed in separate documents, organised by character name and working title. Early printouts were separated into chapters, each one held together with a bulldog clip, so that I could shuffle and reshuffle while reading.

When I finally created the first complete manuscript, I used a random number generator and manually combined the chapters into a single file. I’ve never considered putting together a preferred or canonical order. The thought of it seems a bit…wrong to me. The chronology of the story can be reconstructed in part—some events clearly happen before others—but a grand overarching chronology would be impossible to determine. That’s not how this story works.

At the end of an exhaustive process, I wasn’t sure if I’d succeeded. It wasn’t until the first feedback from beta readers (each of them with their own unique random shuffle) that I suspected maybe this was working as intended. A good indication was that some of these early readers did their own reshuffling to see if I had cheated.

The coding to compile finished print-ready files is done in Automator, the computer equivalent of a Rube Goldberg machine.

The long process of conceiving, planning, and writing Ex Libris has led me to a different way of thinking about raising tension in a narrative arc. The behaviour of the characters introduced in the opening sequences is gradually becomes clearer as their background is revealed. It doesn’t matter in what order those revelations happen.

The best analogy I’ve found is that it’s like a jigsaw puzzle. The order in which you place the pieces doesn’t change the final picture, but it does change how you experience the journey towards it. Adjacent chapters might flow or they might juxtapose. A character might disappear from the story for a while. A particular piece of key knowledge might be revealed earlier or later. The story has a different rhythm between copies. If the traditional narrative arc is the linear curve, this is more two-dimensional.

So does it work? That remains my burning question as I finalise editing and prepare to publish. It’s impossible to speak for every possible combination. There are 479,001,600 of them so I can’t check. It’s something every individual reader will have to determine on their own based on the version of the text they receive. I’ve always hoped that the story might be good enough to transcend its construction. I imagine a reader happening across a copy of Ex Libris, with no prior knowledge of its creation, who will read from cover to cover and enjoy it.

Is that even possible? I guess we’ll see.

The crowdfunding campaign to publish Ex Libris is live until 25 November 2019.

This is an article about an augmented reality practice-based research project, The Fantasia Express, commissioned by the UK Department for Transport through an InnovateUK competition. And about how a non-writer tried to create a story around a train journey from London to Edinburgh.