We are two academics who met in Cape Town when  our children became friends. Conversations around our shared interests in history, technology and Cape Town’s District Six got us speculating about social justice pedagogies in our respective disciplines. Our preliminary discussions revealed  many parallels in our approaches but also intriguing  distinctions, enough to seed a commitment to collaborate. At the time, Siddique was teaching geomatics students who were studying towards qualifications in land surveying and Geographic Information Systems (GIS) at the Cape Peninsula University of Technology (CPUT). David, an archivist, was organising immersive professional experiences for University of Michigan (U-M) graduate students with NGOs in Cape Town through his school’s Global Information Engagement Program (GIEP). In this article we discuss our three-plus years of collaboration to enhance social memory through joining traditional primary sources with emerging information technology tools in District Six, with emphasis on our latest effort to integrate Augmented Reality (AR) with GIS, archives, and storytelling.

District Six in Cape Town is perhaps the most well-known site of apartheid forced removals (some 60,000 Black individuals between 1968 and 1983). It was a cosmopolitan, multi-racial and multi-denominational urban community viewed by the government as a slum that violated the edicts of apartheid’s ‘separate development’ segregation ideology. It was also favourably located close to the city centre, nestled in between Table Mountain and the sea. Upon removal of the residents, almost every building (houses, community halls, shops, public amenities, health facilities, cinemas, nightclubs, hotels, factories, businesses) was demolished by bulldozers, except for a few mosques, churches and schools. Today, the site sits largely unreconstructed and its legacy and future remains deeply contested. The last four months of 2021 saw District Six mired in controversies over redevelopment planning, city evictions of the homeless, restitution and resettlement for ex-residents, and increasingly realised fears that many returning claimants will pass away before securing one of the District’s newly constructed homes (Bantom, 2021; Charles, 2021; Lepule, 2021; Thebus, 2021).  

Our collaboration started in 2018 and builds on the surveying and mapping work carried out by Siddique on the site since 2006. As a much-needed decolonising pedagogy in engineering education, Siddique attempted to conscientise students to the socio-political-historical aspects of the site that their university was situated on. The university, which was originally called Cape Technikon, was built on a large portion of the demolished District Six, and was a Whites-only institution during apartheid. In Cape Technikon’s curriculum, District Six’s troubled history was silenced and buried under the technical language of geomatics. Fieldwork tasks were ahistorical – the rubble under their feet was nothing more than land that needed to be accurately measured and represented on a map. This needed to be changed, and Siddique’s students started to dig up the past, linking mapped locations to what (and who) used to exist there before. 

Figure 1: District Six aerial photographs. Top left – 1953; Top right – 1968; Bottom left – 1983; Bottom right – 2019.

Figure 1 shows four GIS maps of the same geographical space. The red line is the outline of District Six, the area that was declared Whites-only in 1968. One can observe the drastic changes on the landscape. In 1953, District Six was fully developed, with narrow streets and dense concentration of buildings. By 1968, the first signs of demolition were observable. In 1983, District Six resembled a war zone, with most buildings destroyed, and in 2019, much of the district was open space, with the CPUT campus a dominant feature.

Our first efforts in 2018 were consciously designed as an interdisciplinary effort to join geomatics, archiving, and storytelling. We sought to virtually reconstruct aspects of District Six in terms of the lost built environment and stories of still living ex-residents and their experiences before, during, and after demolition. We  brought together a wide range of stakeholders, including ex-residents, St. Mark’s Church (one of the few structures that survived demolition), the District Six Museum, GIEP, and the departments of Town & Regional Planning and Civil Engineering & Surveying at CPUT. St. Mark’s then current rector wanted to foreground the church’s history and reached out to Siddique. At an early project planning meeting at St. Mark’s, we pitched the idea of using its baptismal records and integrating them with Siddique’s mapping efforts and the memories and archives of selected ex-resident parishioners.The baptismal records date from the late 1800s and are a potent symbol and source of District Six’s heritage, documenting individuals, families, social networks, livelihoods and how they manifested across space and time. 

We created a prototype demonstrating that it’s possible to accurately plot the addresses extracted from the baptism records and connect them to the life stories of ex-residents. The prototype selected and mapped approximately 2,000 baptisms between 1950-1958. This was a detailed undertaking that involved onerous human and computing efforts of extracting, checking, and scripting to clean the data. CPUT students created a GIS map that geolocated addresses and integrated it with maps of other sites in District Six. This work represents the first comprehensive digital historical spatial record of District Six that contains discrete and granular locations of a diverse range of important sites (such as businesses, public services, streets, and the homes of baptised babies). Figure 2 shows some of the sites, overlaid on the 2019 map.

Figure 2: Geo-location of specific sites in District Six on the contemporary landscape.

A collaborative exercise with the GIEP team confronted the vagaries of deciphering handwritten baptism entries, confirming and normalising street names and addresses and locations. We worked with St. Mark’s to identify ex-residents who were willing to share their District Six experiences through recorded oral histories supplemented by their own personal archives. Six ex-resident St. Mark’s parishioners agreed to participate. Interviews covered a range of community members. Each participant was photographed with their baptismal record, and copies made of any records they chose to bring in to aid their storytelling, such as childhood photographs and/or mementos. All generated materials from this iteration – including the digitised baptism registries, the oral history methodology, permissions, and oral history recordings – was deposited with the District Six Museum under a deed of gift. (Huang, Leal and Rubin, 2018).

In 2019, we built upon this work with a second GIEP and CPUT cohort. The District Six Museum hosted this effort. In this iteration, we sought to strengthen our oral histories, identify and plot key landmarks on the GIS map, and integrate all of this as a publicly accessible interactive multimedia website. We re-interviewed five of the six 2018 storytellers and added two new ones. We mined the District Six archives to locate archival images and supplementary primary sources of twenty-nine landmarks: eight social centres, a hospital, nine places of worship, seven schools, and 4 cinemas/bioscopes. Finally, we integrated all of this information onto the museum’s website as an online experience that enables visitors to overlay historical District Six with the contemporary landscape, learn storytellers’ biographies through geolocated photographs, documents, and audio, and identify, locate, and learn about historically significant landmarks (Cox et al, 2019). This effort is now hosted by the District Six Museum’s website as the ‘St Mark’s Memory Mapping Project’ (Figure 3). Siddique’s follow-up work, a 21 minute video provides vivid evidence of the fruits of joining geomatics and archiving to tell the story of erased pasts (Motala, 2019). 

Figure 3: Snapshot of the St. Mark’s Memory Mapping Project website.

Over the past two years, Siddique developed a ‘Haunted Walks of District Six’ walking tour, using analog printouts of the GIS map and historical images of the tour’s stopping points. To date, this analog version of Haunted Walks has been delivered to some 200 individuals – academics, students, artists, filmmakers, and ex-residents. The goal is to provide walkers with a deeply affective and transformative experience by integrating the primary sources described above with real time traversing of the contemporary landscape. This approach deploys the ‘counter-surveying’ methodology, which combines land surveying and mapping to find and mark the locations of sites on the physical landscape that no longer exist (Motala and Bozalek 2021). This method can identify the exact locations of demolished homes in the present day for the benefit of ex-residents, as well as to locate other sites to reinforce and graphically illustrate the gravity of what was lost during demolition. 

We now believe that the current analog version of the walking tour needs to be reimagined, strengthened, and experientially enriched with the thoughtful incorporation of digitised and digital content via AR technologies. As an interactive technology that runs in real time and registers and co-locates contemporary and virtual historical objects with one another through handheld devices such as mobile phones and tablets, AR is a logical next step in our collaboration. We are now recasting the Haunted Walks tour as an AR enhanced experience. Users will be directed to the precise locations of important demolished sites in District Six and be shown what used to exist at that location through stories, superimposed archival images, documents, videos and sound clips. AR will enable walking tour participants to immerse themselves simultaneously in both the contemporary and historically erased landscape, essentially co-geolocating themselves across time. 

To continue this work, in 2021 we received a collaborative faculty seed grant from  U-M’s African Studies Program’s African Heritage and Humanities Initiative, running through 2022. This work is directed towards a decolonial praxis based on connecting developments related to our scholarship in social justice geomatics (Zembylas et al 2021) and archival social justice (Wallace et al 2020). It will explore new heritage-related theoretical and praxis directions for engineering, archiving, information technology (AR and GIS), and their intersections in social justice memory work. Work is proceeding along three axes, 1. Theoretical (joining counter-surveying and social justice archiving to probe the intersections of ephemerality, affect, and non-institutionalized memory making); 2. Methodological (development of counter-surveying and memory making and transmission praxis) 3. Socio-technical (incorporation of AR to transform the Haunted Walks experience). We will reinvent the embodied experience of the Haunted Walk, with prospect for application to other contexts of forced removal, recovered and contested memory, and ongoing justice-seeking and activist efforts (acknowledgment, memorialisation, restitution, and reparations). 

However, we also take seriously the advice of Tuck who warns against such work as being solely aimed at the documentation of damage. Whilst damage-centred narratives are often collected and deployed benevolently, the resulting work provides a one-dimensional view of Blackness and disenfranchised communities (Tuck & Yang, 2014). We seek to meaningfully contribute both to more fine-grained understandings of apartheid planning practices and the experiences of those subjected to them. Instead of solely focusing on the documentation of damage, we are equally ‘concerned with understanding complexity, contradiction, and the self-determination of lived lives’ (Tuck, 2009, p. 416). We also want to attract a younger generation of technologically adept South Africans, which is especially critical in light of the nation’s ongoing efforts to decolonise its curricula and grapple with its apartheid past and its afterlives. Through this community-based work, we seek to leverage creative technologies, stimulate affective responses in walker-users, and assist students and researchers in reimagining the historical remnants of the ex-apartheid city and its continued effects on the contemporary urban landscape.

To date we have laboured to reimagine and reconstruct the walking tour as an AR app by plotting out specific notable and not well known points of interest. We are  collaboratively working with the District Six Museum’s archives to identify photographs that can be precisely located and superimposed onto the contemporary landscape. We have ideated and envisioned the user experience via a 2.5km walking loop and how that will manifest as virtual content delivery and interactivity via historical narratives and stories and locational positioning in the real world. We have identified necessary pragmatics of pre-tour set-up (route and tour points, disclaimer and liability waiver, operationalising the tour on users’ devices, interface design, location awareness, and toggling between the historical and contemporary landscape), user orientation to and experience at tour points (navigation, multimedia content display of text, audio, visual and moving image, overlaying historical photos on the contemporary landscape), and ending (feedback, sharing on social media, contact information). 

The biggest early challenge has been identifying and evaluating the tradeoffs on alternative AR platforms, given due consideration of the constraints facing prospective users in District Six: device requirements and operating system; navigation; data usage, and; feasibility to realise the completeness of our vision. We want a user to be able to use an Android smartphone or tablet to download our app and be empowered to take a self-guided tour. Project staff have explored AR possibilities mindful of these needs. We have evaluated the status and capabilities of available AR tools / applications along the following parameters: functionality; usability; learnability; incorporating multimedia content and geolocational data; content management; open source vs proprietary; costing; editability, and; prospects for persistence (longevity). After carefully reviewing nine alternative possibilities, we are using Unity3D as our AR authoring environment. It will allow us to integrate our GIS maps, develop our own user interface, activate the user device’s GPS for navigation and camera for transposing historical photographs onto the contemporary landscape.

Parallel to the AR effort, project staff are modifying the current analog version of the tour by developing a curated digital repository of existing and new archival and other supporting primary source materials. This will strengthen and expand walking tour content. This will be supplemented by the development of new and expanded tour narratives / storytelling. For example, we envision that one of the points that will be visited through the app will be Horstley Street (Figure 4), which was an important street in the history of District Six. The buildings of Horstley Street were completely razed, but the original cobblestone street remains.

Figure 4: Horstley Street before demolition (left, District Six Museum archive) and today (right).

Ultimately we value the expansion into AR not as technology for its own sake but rather as a tool to create powerful affective embodied experiences for walkers-users. It is one thing to see the remnants of District Six as pins located on digitised historical and contemporary aerial photographs, as was done with our previous work. We believe the embodied and embedded process of walking across the contemporary landscape to visually ‘see’ what used to be there provides a more tangible, meaningful, and moving experience that will create a lasting impact. We anticipate launching the app in 2022.

Bibliography

Bantom, K (2021), ‘D6 ‘caretakers’ named,’ news24, 14 September [online]. Available at: https://www.news24.com/news24/SouthAfrica/Local/Peoples-Post/d6-caretakers-named-20210913 (Accessed 13 December 2021) 

Charles, M (2021), ‘City of Cape Town evicts homeless in District Six during massive operation,’ news24, 20 September [online]. Available at: https://www.news24.com/news24/southafrica/news/watch-city-of-cape-town-evicts-homeless-in-district-six-during-massive-operation-20210920 (Accessed 13 December 2021) 

Cox, V, Dall, V, Qui, V and Yelk, J (2019) ‘St. Mark’s memory capture and interactive mapping.’ (Global Information Engagement Program, School of Information, University of Michigan)

Huang, J, Leal, M and Rubin, M (2018) ‘Final report: St. Mark’s community heritage and baptismal record project.’ (Global Information Engagement Program, School of Information, University of Michigan)

Lepule, T (2021) ‘Claimants dying off before ever returning to District Six,’ Weekend Argus, 28 November [online]. Available at: https://www.iol.co.za/weekend-argus/news/claimants-dying-off-before-ever-returning-to-district-six-b8f3ed06-866c-4185-82eb-a9ac13c67d0c (Accessed 13 December 2021) 

Motala, S (2019) “District Six and CPUT: a carto-story.” Available at: https://www.youtube.com/watch?v=iEzydbcVWV4 (Accessed 13 December 2021) 

Motala, S and Bozalek, V (2021) ‘Haunted walks of District Six: propositions for counter-surveying,’ Qualitative Inquiry [online]. Available at: https://doi.org/10.1177/10778004211042349 (Accessed 13 December 2021)

Thebus, S (2021) ‘Plan to resettle District Six claimants plods along,’ Cape Argus, 16 September [online]. Available at: https://www.iol.co.za/capeargus/news/plan-to-resettle-district-six-claimants-plods-along-113c2d18-f4b8-48f4-a6fe-eefa89235742 (Accessed 13 December 2021)

 Tuck, E (2009) ‘Suspending damage: A letter to communities’. Harvard Educational Review, 79(3), 409–427. Available at:  https://doi.org/10.17763/haer.79.3.n0016675661t3n15 (Accessed 14 December 2021)

Tuck, E and Yang, KW (2014). ‘R-words: refusing research’ in Paris D and Winn, MT (eds.) Humanizing Research: Decolonizing Qualitative Inquiry with Youth and Communities. Thousand Oaks, California: SAGE Publications, 223–248. Available at:  https://doi.org/10.4135/9781544329611.n12 (Accessed 14 December 2021)

University of Michigan (undated) ‘Engaged learning,’ [online]. Available at https://engaged.umich.edu/engaged-learning/ (Accessed 13 December  2021)

Wallace, D, Duff, W, Saucier, R, Flinn A (eds.) (2020)  Archives, recordkeeping & social justice. London: Routledge

Zembylas, M, Bozalek, V, and Motala, S (2021) A pedagogy of hauntology: decolonising the curriculum with GIS. In Bozalek, V, Zembylas, M, Motala, S, and Hölscher, D (eds) Higher education hauntologies: living with ghosts for a justice-to-come Oxon: Routledge, 11-28). DOI: 10.4324/9781003058366 

Children’s literature has always been a genre curious to experiment and play with media. Just think of pop-up-books and how the Alice in Wonderland adaptation Alice for the iPad (2010) by Atomic Antelope was one of the first apps to explore the features of the iPad. Since the introduction of the iPad in 2010, several attempts have been made to explore meaningful alliances between the print book and the digital device using augmented reality technology; creating hybrid experiences combining the traditional medium for children’s literature and its newest carrier. Such experiences depend on the user installing an app on her digital device and pointing its camera at the pages of a book. The device reads or decodes the data on the paper page and activates and displays content on the screen. The user’s physical, multi-sensory and cognitive engagement in such experiences is also one of complex character as she is managing two technologies at the same time. 

On the one hand, the book or codex technology which include turning the pages and navigating in specific ways, and on the other hand, the digital device and its specific interface navigation. In this way, and as is the defining nature of augmented reality, the book that exists in the user’s real-world environment is enhanced by computer-generated sensory information thus playing with the user’s perception. Off-hand, the effort to create this union seems slightly paradoxical if we consider the fact that the iPad was conceived and designed specifically as the unification of the book and the computer. So, what can actually come of this persistent Sisyphean task of making the print book and the tablet computer work together? 

This article will dig into this question by taking you through some remarkable international examples of literary AR book projects for children, their development over the past decade and the experiences they can produce. It will then shift from an international to a more local, Danish perspective and explore the potential of AR books for children for supporting reading motivation. In Denmark, a new partnership saw the light of day in 2021 in the company Smart Books. The company consists of the popular YouTuber and influencer Rasmus Kolbe, best known under his old boy scout name Lakserytteren (directly translated: the salmon rider) and Søren Jønsson, who is a successful and experienced producer of games for children. Smart Books deliver an augmented reality ‘smart book’ concept, where the reader chooses the path through the paper book’s narrative, interacting with both a book and digital content on a smartphone or tablet, and in this way gains an interactive reading experience. While this strategy is new in a Danish context, the venture also stands on the shoulders of a line of previous AR book projects.

Literary “AR + book” projects for children

When looking over the last decade’s projects that combine AR and paper books aimed at children, it is clear that this media interplay has gained a stronger footing in non-fiction and educational publications than in more literary, narrative projects. Generally, in these latter projects there has been a development from early projects that mainly ‘digitize’ the content of paper books, such as 3D animation of characters in picture books (e.g., Resin’s Two Left Feet, 2013), without asking for the user’s engagement to any significant degree, to newer projects that play with the potential of the augmented digital environment more fully and call for the user’s engagement via interactive game elements (such as Books & Magic’s The Little Mermaid, 2016). However, in some of the latter projects the print book’s materiality and role is in turn neglected. The crux of the matter seems to be to find a balance between the media where one is not a gimmicky appendix to the other. 

Resin’s Two Left Feet

Books & Magic’s The Little Mermaid

If we look to projects that can be characterised as literary in the sense that storytelling and an aesthetic experience are at the forefront of the works, projects produced  by the now hibernating American multi-platform storytelling company Moonbot Studios stand out. These works include The Fantastic Flying Books of Mr. Morris Lessmore and The Numberlys. Both works are aimed at children, The Numberlys at young children (3-7 years old) and Lessmore at slightly older children, yet a target audience is not mentioned anywhere. They both exist in several media at the same time constituting cluster works (Mygind 2017): as picturebooks and AR apps that can be used in conjunction with the picturebooks, as standalone interactive apps and animation films. In the changeable, fleeting world of apps, these works are already old (the books, the apps and films came out in the period 2011-2014) and are not available for purchase anymore, but this does not mean that they are not worth mentioning here. On the contrary, these works draw closer to a balance and a mutual relationship between the print and digital media than many other works. 

Moonbot Studios’ The Fantastic Flying Books of Mr. Morris Lessmore, teaser for all versions

Moonbot Studios Imag-n-o-tron: Numberlys edition

The picturebook The Numberlys takes full advantage of the book medium by playing with the reader’s ingrained expectation to the book by using the oblong format and mixing the reading directions of the book. The book is bound in the short, left side, which would normally mean that the book is read horizontally, but already from the title page the expectation is denied as this page must be read vertically and the book must, therefore, be turned. The title is one large image that spreads from top to bottom, with the five little main characters marching across the page at the bottom. This vertical reading direction enhances the impression of the vast, oppressive world of all-pervading numbers that the characters inhabit. This is a characteristic of the picturebook that it takes advantage of the book medium, its materiality and reading conventions and plays with them to convey meaning. In relation to the hybrid AR book experience, it is noteworthy to point out how the interactivity between the reader and the medium becomes part of the way meaning is conveyed in this universe and therefore not something that is reserved for the digital component.  

In relation to the rest of the cluster work that The Numberlys compose through its many independent media versions, the AR app IMAG-N-O-TRON: Numberlys Edition is the only one that cannot stand alone. Since the app is dependent on the picturebook to activate its content, it can be characterised as an intracompositional transmedia phenomenon (Dena 2009), meaning that the ‘AR app + book’ composition is transmedial in itself, and the app is not a self-contained narrative unit. When opening the AR app, the reader is transformed into and staged as a player, collector and detective, using the digital device as a magnifying glass through which to explore and investigate the picturebook. The AR app encourages the player to scan the book for objects, which,  when located on the paper page via the camera, will turn into animated objects on the screen and be stored in the apps interface. When the objects from the book have been collected, the player can build new, fun constructions in the digital space and practice constructing letters and numbers. In this way, the app encourages the player to perform creative, educating tasks that mimic what the fictional characters do in the picturebook, thereby extending what we might call the core values or message from the picturebook to the digital environment via the AR technology.  

In relation to the aforementioned balance between the media in AR book compositions, it is noteworthy how, on the one hand, The Numberlys picturebook composes a self-contained narrative entity, while, on the other hand, the ‘AR app + book’ composition actually works independently as well. Of course, the reader/player will gain a deeper, more informed experience if she reads the story in picturebook, but it is not a prerequisite to engage with the part of the Numberlys universe that is available in the AR book composition as they offer two distinct kinds of engagement: engagement in a narrative and engagement in playful activities.

Disrupting reading and media cultural hierarchies

Moonbot Studios’ few productions were, and still are, innovative and remarkable examples of AR + book compositions and, on the whole, of experiments with multi-platform, transmedia storytelling, however, the venture did not continue and did not set a precedent for subsequent international AR books for children. If we look to the recent Danish Smart Book concept, this endeavour is targeted at older children, specifically children from 9-13 years old, and here we find yet another approach to the composition of the AR book. 

The Smart Book concept consists of a series of paperback books with individual accompanying AR apps. Currently, three books have been published in Danish and two in English are forthcoming. The series takes place in a fantasy universe of wizards and magic where the reader assumes the role as the protagonist “you”, the First Student of the Firemaster. Just like in the so-called gamebook series for children from the 80-90s Choose You Own Adventure, you navigate the book by reading short numbered chapters that present you with a choice and, depending on the choice, directs you to a new chapter. Some chapters also present the reader with challenges and puzzles, often small maths related puzzles, that must be solved to move on. These obstacles are presented in the book via simple illustrations and text but must be met and solved in the AR app that, when accomplished, will direct the reader to a new chapter in the book. The content of the puzzles is most often not related to the narrative and therefore not narratively motivated. Instead, they offer different kinds of cognitive engagement.       

An interesting aspect of this AR book composition is that the traditional way of reading a book is turned into something else via the non-linear and unknowable reading path. We might say that the book and the AR app create both a material and fictional space in which the reader moves back and forth, yet with a feeling of moving forward without knowing when the journey will end. Normally the reader of a book can see, feel and count the number of pages read and the number of pages remaining in the book. This conventional way of navigating a narrative in a book is suspended and disrupted both by integrating the AR app and through the non-linear structure resulting, paradoxically, in both a higher degree of agency assigned to the reader and a higher degree of obscurity or mystery for the reader.

In their public presentations of the Smart Book concept, the two owners of the company, Søren Jønsson and Rasmus Kolbe/Lakserytteren, associate their project and their motivation behind it with the widespread concerns about the decrease in reading among children. A Danish study of children’s reading habits conclude that children’s joy of reading literature decreases with age and that the drop happens between 5th and 6th grade (Hansen et al. 2017). This situation is the point of departure for the Smart Book project, and the owners’ stated mission is to reignite children and young people’s joy for the written word, and showing them the way to the AR books especially through Lakserytteren’s media channels, such as YouTube, TikTok and Instagram.

The Smart Book series adapts the known “choose your own adventure” form to the AR book, hereby disturbing existing notions of what it means to read literature and proposes a new way of reading. These hybrid works of fiction combine print and digital media in a mutually dependent fashion that challenges prevalent public debates about onscreen reading. In these debates a notion of a media hierarchy is formed. A hierarchy that contrasts print media as the authentic, educational way of reading with digital reading (both visual and audio) as the inferior way of reading. Transmedia AR book projects have an intrinsic potential of being able to break down this hierarchy and offer new multi-sensory, transmedia experiences that support different reading styles. 

References

Dena, C. 2009. Transmedia Practice. Theorising the Practice of Expressing a Fictional World across Distinct Media and Environments. PhD dissertation.

Hansen, S. R. et al (2017): Børns læsning 2017: En kvantitativ undersøgelse af børns læse- og medievaner i fritiden, Læremiddel.dk: Tænketanken Fremtidens Biblioteker.

Mygind, S., 2017. “A Chinese cluster: Danish-born digital comic as source for transmedia design and innovation” in Ensslin, A. et al (eds.): Small Screen Fictions. Vashon Island, Washington: Paradoxa

The Writing Platform is looking to commission articles of approx 750 to 2,000 words on creating stories with augmented and virtual reality technologies. This might include the use of AR and VR in fiction and non-fiction literature, journalism, theatre, movies, and games as well as articles that explore AR and VR as tools to promote existing works or as alternatives for live social events (e.g. face-to-face meetings with authors). 

We are looking for papers that examine forms such as: AR stories in which you become one of the protagonists (e.g. Wonderscope), VR experiences in which spoken and written word plays an important role (e.g.The Chalk Room) or VR poetry and VR experiences that blend spoken poetry with dance (e.g. VR Nightsss), AR books and comics (e.g. Modern Polaxis), VR theatre and opera (e.g. The Under Presents: The Tempest), performances that blend VR with interaction between the performer and spectator (e.g. Draw Me Close), holographic theatre (e.g. Chronicle Of Light Year) and live augmented reality glasses performances (e.g. Verrat der Bilder), as well as AR and VR games (e.g. Pokémon Go! or Virtual Virtual Reality). We are also interested in articles on VR experiences that can be used for therapeutic purposes (e.g The Wayback), VR and AR documentaries (e.g. The Waiting Room), VR and AR literary adaptations (e.g. Metamorphosis VR) or VR literary archives (e.g. Digital Fiction Curios), and many more. 

We are interested in the ways in which technologies can produce new forms of storytelling and are also keen to receive articles focusing on expanding and diversifying audiences for immersive storytelling experiences as well as on the ethics of using new technologies and platforms in storytelling. 

The deadline for submission of proposals and ideas for articles has been extended to 1st November 2021.  Articles will be published on The Writing Platform website late 2021/early 2022. Once your proposal is accepted we will negotiate a deadline for the full submission with you.

Your proposed article might fit into one of the following categories: 

Resource: for example, a how-to guide for practitioners about creating stories with any AR/VR tool

Research: for example, an overview of a collaborative research project on AR and VR in storytelling or an examination of the impact of using VR and AR on enhancing audience immersion 

Experience: for example, an account of a VR and AR experience that you have experienced or developed

News: for example, highlighting a new project or opportunity in the VR/AR field that our readers might not have heard of before

Projects: a case study of an especially innovative or inspiring project with impactful outcomes 

We have a small commissioning fund for freelancers (£100 per article).

To propose an article, email Agnieszka Przybyszewska (a.przybyszewska@bathspa.ac.uk) with a 100 word overview of your idea and tell us which category or categories it would best fit into. 

Way back in the wilds of the year 2008, artist-extraordinaire James Morgan and I engaged in an animated discussion about Augmented and Virtual Reality. At that time James and I were collaborators-in-crime in the Third Faction Collective, a group of digital artists intent on constructing game interventions in Massively Multiplayer Online Spaces. During this discussion, I pitched to James an idea to establish an online space devoted to all things Synthetic Reality based (my umbrella term for Virtual Reality, Augmented Reality, and Mixed Reality). This space, called Augmentology 101, intrigued James to the point where a decision was made to sponsor it through the Ars Virtua Foundation and CADRE Laboratory for New Media. What followed was an amazing exploration into the creative potentials of Synthetic Reality – what’s now known as XR (Extended Reality) – and how it might manifest within the realm of electronic literature.  

It’s now been 10 years since the initialisation of the Augmentology 101 project. During this decade, there’s been a major upswing in VR and AR production and development, with impactful XR content such as Firebird – La Péri (a 2016 English/Chinese/French multilingual VR Experience) and Queerskins VR (2018) being standout examples.

Screenshot from the 2016 Multilingual Virtual Reality Project “Firebird – La Peri”

My own attempts at merging digital literature into developing XR fields have been multiple and varied, originating in delving into VR in the 1990’s when VRML was the shiny new thing. Surprisingly enough, the creative and technical challenges that VR creators faced back then are similar to those we face today: high performance requirements, mainstream adoption hurdles (see: Gartner Hype Cycle), and monetisation dilemmas are all relevant. Likewise, skillsets required by VR content creators in the mid 1990’s again parallel XR creators of today, including developing a deep knowledge of spatial storytelling logistics; emotional intelligence; and the ability to formulate story experiences that take into account various hardware and platform limitations such as field of view constraints, tethered headsets restricting natural movements, and hardware specific limitations like the screen-door effect.

In terms of XR projects I’ve produced in the last decade, a brief selection includes conceiving of and co-developing the 2013 anti-surveillance AR game #PRISOM, and in 2015 mapping out with Andy Campbell the (now unfinished) PC/VR project Square Ebony that was to be filled with: “…movement/imagery like huge ‘Panic Room’ landscaped letters…a force field of green…branches intertwined…tangles being text…[that] revolves around an entity…this entity is slowly reconfiguring itself…at the top of a hill/mountain/plateau surrounded by brackish water” (notes from Square Ebony Project Meeting and Documentation, Breeze and Campbell, March 10th 2015). In 2016 I lectured as part of the “Future Possible: Beyond the Screen” Series which centred on how VR can transform creative practice, and which also included a live VR performance walkthrough using one of my Tilt Brush created works. In 2017 I keynoted at the Electronic Literature Conference with a VR performance presented both live at the Conference and simultaneously in Virtual Reality.

“Layering the New real: Tracking the Self in Disembodied [Un] Virtual Spaces” Keynote

Press Image for “Our Cupidity Coda”: VR Literature

Thorough participation in a high-end VR based experience like Perpetual Nomads hinges entirely on immersion, which is triggered initially through the audience having to don gear that firstly reduces their ability to engage in their actual physical space in standard ways (their vision and hearing being “co-opted” into a VR space). The leap of faith the audience needs to make to establish a valid and willing suspension of disbelief (as Samuel Coleridge so aptly phrased it) is already set in motion by the fact a user is entirely aware from the moment they slip on a VR Headset that their body is in essence hijacked by the experience (haptically, kinetically), as opposed to a more removed projection into a story space via more traditional forms (think book reading, movies, tv). Such body co-opting might lead a user to disengage from the VR experience from the very beginning which will reduce the likelihood of true immersion: alternatively, they may readily fall headlong into the experience with an absolute sense of engagement and wonder (the preferred option as a VR content creator!) if the work has been precisely crafted.

For the most part, XR projects such as those mentioned above currently exist only in the mainstream margins, with a majority of experiences requiring costly high-end VR rigs and expensive desktop computers that demand audiences experience the works in their optimal state. To counteract this selective catering to the exorbitant end of the XR market, in early 2018 I had the idea to create a VR Experience that would reduce the mandatory use of high-end tech. This project would instead cater directly to a range of audiences by crafting a work that could be experienced across a far larger (and much more accessible) range of lower-end tech. This VR Literature work is called A Place Called Ormalcy.

Title Image from the “A Place Called Ormalcy” Press Kit

A Place Called Ormalcy is digital literature designed for, and developed in, Virtual Reality. It was constructed using the Virtual Reality Application MasterpieceVR to craft the 3D models, with each chapter (made up of 3D models, text, and audio components) then combined and hosted via the Sketchfab platform. It’s comprised of a text-based story made up of seven short Chapters which are housed in 3D/Virtual Reality environments. It can be accessed via a wide range (crucial in terms of its social commentary aspect) of mobile devices, desktop PCs and both low-end and high-end Virtual Reality hardware. Audiences using the cheapest type of VR equipment (such as Cardboard headsets) are able to access complete versions of this VR literature experience, as are users of any net connected mobile device with a WebVR-enabled browser.

The story of A Place Called Ormalcy (warning: spoilery parts ahead) unfolds through a series of snapshots of the life of Mr Ormal, a happy-go-lucky law-abiding chap who resides in the aesthetically cartoonish world of Ormalcy. Ormalcy exists in an alternative universe complete with its own idiosyncratic language patterns. The storyworld initially presents as a Utopia full of innocent “claymationesque” contented creatures and happy citizens. As the story plays out, however, it soon becomes apparent that in actuality, this VR Experience allegorically traces the makings of a dystopic society, and how such fascist principles can arise in the most benevolent of places. This VR Literature work has social commentary at its very core, commenting directly on and about the rise of current totalitarian trajectories and the contemporary malaise, confusion and accompanying acclimatization patterns.

“A Place Called Ormalcy” Chapter Progression

A Place Called Ormalcy uses a combination of WebVR, 3D, VR, text and audio assets in ways that mirror a slow dystopian creep. In the desktop and mobile versions, each chapter becomes progressively visually cloistered, with dark fog and grainy distortions increasing to finally create a type of gun-barrelled claustrophobic effect. This combines with a gradual leaching of the intense colours found in the free-flowing organic imagery of the initial Chapters which results in a startlingly stripped back, fuzzy palette and model constructions: vibrancy gradually bleaches out to stark black, white and greys. Correspondingly, the 3D tableaus and audio tracks likewise alter from an initial complexity – Mr Ormal begins his story journey waving directly to the audience in “Chapter Wonne” in a bright and blooming space – which incrementally shifts towards the dramatically minimal in the final “Chapter Severn” where Mr Ormal transforms into (…spoiler alert here…) something vastly other.

In the VR version of A Place Called Ormalcy, additional effects mark the dystopic “boiling frog” dilemma that Mr Ormal faces. Each VR tableau subtly increases in size and scale as the Chapters progress, with the audience finding themselves in the climatic Chapter in a looming monochromatic set surrounded by huge windowless block-shaped buildings devoid of detail – except multiple, and menacing, “88” shaped logos (and the awfully transfigured Mr Ormal). In the VR version, the text becomes increasingly difficult to navigate, with the audience having to teleport, twist and turn in the VR Environment to read each annotation, echoing the “fake news” proclamations of our contemporary Western world where it is becoming increasingly difficult to access truth over relentless propaganda.

Although A Place Called Ormalcy may seemingly convey a message of hopelessness or helplessness, the ending does contain clues that all is not lost in this particular dystopian scenario – the final soundtrack offers hope, with protestors chanting and proclaiming resistance as key. Just as VR Literature can work to extend the creation of accessible electronic literature beyond the text-centric to truly encapsulate the haptic and the spatially-oriented, A Place Called Ormalcy illustrates how XR projects can act as relevant social commentary at a time when it is sorely needed. I look forward to continuing to promote, create, and experiment with stretching the limits of VR and AR while producing XR projects that are openly accessible, as well as socially relevant.