For the last few years I have been experimenting with telling complex, challenging and nuanced historical stories using creative technology. In September 2022, I received a fellowship from Bath Spa University’s Narrative and Emerging Technologies (NET) Lab to explore ways in which to use immersive and emerging technologies to tell stories within some 19th Century family letters and diaries. I have a background in documentary filmmaking and in writing for theatre. More recently I have been working on large R&D projects aiming to promote the use of immersive technology in the creative industries, so the fellowship seemed like the perfect opportunity to try and marry the three areas together. 

Photograph of some of the nineteenth century letters and diary. Credit: Rachel Pownall

Initially, I spent time exploring different types of immersive narrative experiences. I looked at everything from one of the highly commercial immersive Van Gogh exhibitions to VR experiences like Randall Okita’s The Book of Distance to the immersive audio piece, Intravene, by Darkfield. Alongside this I spent time getting to grips with the source material. As well as examining the family documents in detail, I had to do a lot of detective work to try to fill in the narrative gaps; delving into archives, consulting experts, accessing online census and church records then re-examining and interpreting family stories in the light of new information thrown up by this research. 

While there were several fascinating narratives contained within the family letters and documents, I focused mainly on the story of my great, great grandmother Charlotte Coostriah Denman who was half British and half Indian. She was brought from Maharashtra, India to North Wales in 1833 at the age of three by her father William Denman who was an Ensign in the East India Company’s army. She was told that her mother, who died giving birth to Charlotte, was Spanish and she grew up knowing nothing of her Indian heritage. 

As a white British woman, albeit a direct descendent of Charlotte Coostriah, I am very aware of the cultural sensitivities around telling this story. During my fellowship, I was lucky enough to have Dr Priya Atwal from the University of Oxford, as consultant. Her research specialism is cultural politics of empire, particularly across Britain and South Asia in the 19th and early 20th Century. She was able to advise on the historical and cultural ramifications of the East India Company’s colonisation of India and acted as script consultant as I pieced together and started to tell Charlotte’s story and that of her mother. In addition, in order to understand the cultural norms in the UK at the time, particularly relating to women’s experiences, I consulted with Dr Jackie Collier from Bath Spa University. Her research specialism is the social political and cultural history of the long eighteenth century in Britain with a specific interest in gender. 

My research unearthed a huge amount of information including some revelations. With Dr Atwal’s help, I discovered that a mistranslation of some blurry 19th Century writing meant that Charlotte’s mother was not called Lasiml Coostriah as the family had always believed, but Laksmi Coostriah, a name that gives a much stronger indication of her cultural and religious background. I also uncovered parallel stories relating to other women whose lives contrasted with those of Charlotte and Laksmi but were also bound by the strict class, religious and gender conventions of the time. For example, Charlotte’s Aunt Elizabeth was forced to renounce her fiancé, Capt John Molyneux, because his wealthy and powerful father threatened to disinherit him if they went ahead with the wedding. She never married and went on to play a big part in bringing up the orphaned Charlotte.  

I found Twine really helpful in structuring the different narrative strands and offering alternative ways through that focused on different areas.

Screenshot showing the complex narrative structure of one overall storyline in Twine

I had a small amount of money as part of my fellowship to pay for a Proof of Concept (POC) prototype and chose to create one using Augmented Reality (AR) with smartphones because I wanted the story to be as accessible to as many people as possible. I commissioned the AR & VR studio Zubr to create an Instagram filter that would tell a simplified version of William Denman’s journey to India and his subsequent return to Britain with the infant Charlotte.

Screenshot showing simplified narrative structure of prototype in Twine

The ultimate aim was to create a museum-based exhibit where the public would move from element to element triggering sections of the story using their smartphone and acting as detective to uncover the hidden truths behind them. The POC prototype was a first step to test if this was possible.  

For the prototype, we used a mixture of audio that included quotes from some of the letters, music and sound effects, and visuals like the boat, The Lord Lowther, that William sailed on to India and a 19th Century sketch of a Spanish lady in traditional dress (purchased from archive companies). 

Ship Lord Lowther engraved by E. Duncan England. Credit: Peter Horee/ Alamy Stock Photo

Costume of a Spanish lady, 19th century. Credit: Florilegius/ Alamy stock photo                               

Artwork by Bhavana Ram Mohan

We also used elements from some beautiful mood boards that I commissioned from artist-research and illustrator, Bhavana Ram Mohan, whose work explores decolonial curatorial and heritage practices.

The story elements were triggered by holding a smartphone over an object, in this case one of Elizabeth’s diaries. While it was an interesting process to distill the story into five simplified elements, ultimately the prototype did not work as a POC. The audio elements had to be less than two lines of script, and it was just not possible to do justice to so complex and nuanced story in such bitesize pieces. The story demanded a more narrative led approach that can only really be achieved, I now believe, by using an audio first approach. 

Photograph of Elizabeth Denman’s diary from 1833. Credit: Rachel Pownall

Since attempting this prototype, I have looked at other ways to tell the story. This includes plans to create an interactive AI driven chatbot using RocketMaker’s PORTRAIT technology, which would enable participants to converse with a virtual avatar of Charlotte about her life. However, our application to the MyWorld More than AI Sandbox fund was unsuccessful.  

Subsequently, I have written a couple of monologues from the perspective of Charlotte and her Aunt Elizabeth and I am currently looking at how to create an audio led immersive, interactive VR experience to tell their stories. I have put together a diverse team of creatives to achieve this but gaining funding is the most difficult issue with trying to create such an experience. In the meantime, I am looking at low-cost DIY ways of telling the stories. Recently I have started to learn to code in Unity and I am intrigued about the possibility of including elements of gameplay in the telling of the stories. This is one more avenue to investigate as I continue to look for interesting and engaging ways to explore these narratives using different types of creative technology. 

The following is an excerpt from Amaranth Borsuk’s The Book, part of the ‘Essential Knowledge’ Series from MIT Press. This chapter explores the various ways writers and artists for more than a hundred years have approached the book as an object and a structure that can be cut up and rearranged as early examples of ‘digital’ experimentation.

While we might presume the ability to rearrange a book’s parts is an affordance reserved for the digital realm, artists’ books showcase several historical forms that turn the book into a recombinant structure, allowing readers to create new juxtapositions within it. Such interactivity is present already in the accordion book, which, as an intermediate point between scroll and codex, allows readers to open one spread at a time or unfold several, seeing across the folds’ peaks and valleys to survey the text.

Sonia Delaunay, Blaise Cendrars, 1913, La prose du Transsibérien et de la Petite Jehanne de France, illustrated book with watercolor applied through pochoir and relief print on paper, 200 x 35.6 cm, Princeton University Art Museum. (Detail)

The ability to completely open this structure makes it especially useful for topographic work like Blaise Cendrars and Sonia Delaunay’s 1913 collaboration La Prose du Transsibérien et de la Petite Jehanne de France (The Prose of the Trans-Siberian and of Little Jehanne of France), a vertical cityscape of colorful pochoir paintings and poetry where the eye’s traversal of word and image suggest the simultaneity of a dark past and a vivid present for the poem’s speaker as he recalls a railroad journey from Moscow to Harbin during the Russian-Japanese war of 1905; or like Ruscha’s Every Building on the Sunset Strip, which allows a kind of armchair tourism across the Los Angeles landscape. The form lends itself to exhibition for this reason—we can see more of its contents at a glance than a codex if the accordion is stood on end and extended, revealing every peak and valley, front and back. When the accordion’s ends are attached to a cover, it creates a loop, potentially inviting us to start again. But the accordion need not be a linear or landscape experience. It also permits new juxtapositions by allowing readers to refold peaks into valleys and bring distant pages close to one another. Artists’ books in accordion form remind us that the book is, as Stewart notes, “Western culture’s first interactive medium.”

Every Building on the Sunset Strip

This recombinant quality of the book takes place not only across but within the page. The technique, in fact, appears in some of the earliest movable books, which use volvelles, turnable discs affixed to the page with a pin or piece of string, to facilitate calculation and navigation. The earliest volvelles, those of thirteenth-century Catalan mystic Ramon Lull, precede print, and the technique rose in prominence during the incunable period for its scholarly utility. The Regiomontanus Kalendarium (1476), for example, also included volvelles for astrological calculation. Another important recombinant tool appears in the sixteenth and seventeenth centuries in the form of flap books or turn-up books composed of a printed page with a sequence of flaps that alter the narrative each time the reader lifts a hinge. Also known as transformation books or Harlequinades, for the London pantomime figure they often depict, such eighteenth-century novelty books were among the first marketed to children (by London bookseller Robert Sayer around 1765) offering morals and lessons through the transformations they depicted. The harlequinade’s legacy continues in children’s mix-and-match books that use sliced pages and a spiral binding to allow one to swap a face’s features, create hybrid bodies, or otherwise interchange an image or text’s parts.

The recombinant form lends itself to text as well. French author Raymond Queneau (1903–1976), inspired by such childlike “têtes folles” and intrigued by the possibilities offered by a series of cut pages hinged along a spine, composed fourteen Petrarchan sonnets with the identical rhyme scheme, bound them, and sliced the lines apart. Published in 1961, Cent mille milliards de poèmes (One hundred thousand billion poems) offers the reader 1014 different poems, accessed by turning the lines one at a time to make new texts. To read them all, Queneau calculated, would take more than two hundred million years of devoted study. The work is thus a conceptual one but also offers a pleasurable reading experience borne of the novelty inherent in using the author’s text to generate new poems. No wonder, then, that this work is popular with coders, whose digital implementations enact its computational potential. Such remediations, however, lack the tactile pleasure of the interlocking strips that compose the book. They also cannot replicate the sense of potential made palpable by seeing these strips in front of you, lifting themselves away from the spine of the open book and fluttering apart.

Cent Mille Milliards de Poèmes

Queneau joined forces with a group of French writers in the 1960s who were interested in creating new literary forms based on scientific and mathematical principles, and this text is seminal to the movement. Dubbed Oulipo, short for Ouvoir de Littérature Potentielle (Workshop   of Potential Literature), the group pioneered constraint- based writing, which set up a rigid conceptual basis for the production of a work, but one that could yield any number of potential results. Cent mille milliardsis rife with potential, and the interactivity through which we  activate that potential, while it gives some agency to the reader, also highlights Queneau’s authorial genius. The task of com- posing interchangeable sonnets in the identical meter and rhyme scheme draws attention to his authorship, as does much Oulipo work, including Georges Perec’s La disparition (Editions Denoël, 1969), a novel composed without the letter “e” that provides a parable for the disappearance of millions of Jews, including the author’s own parents, during the Second World War; and Anne Garréta’s Sphinx (Grasset, 1986), which remains silent throughout about the gender of its protagonist. Members of Oulipo would go on to generate recombinant and computational poetry under the auspices of Alamo, short for Atelier de Litté- rature Assistée par la Mathématique et les Ordinateurs (Workshop for Literature Assisted by Mathematics and Computers), founded by Paul Braffort and Jacques Roubaud in 1981.

Such game-like recombinant texts are not limited to artists’ books, of course. Many of us enjoyed interactive books published for a mass audience in the 1970s and 1980s. These multisequential books, perhaps the best known being the Choose Your Own Adventure series, offered the reader a series of vignettes, each followed by a choice about what to do next. One path through the book led to the best of all possible endings, while the rest led to trouble, heartbreak, even death. These interactive books—while suggesting that there are many paths, but that we, like Robert Frost, cannot travel them and “be one traveller”—actually allowed readers to pursue them all, thanks to the ability to bookmark the choice point with a finger or slip of paper and read each of the potential outcomes before moving on. One such book, Inside UFO 54–40, took advantage of readers’ tendency to cheat by including a page spread inaccessible through any of the reading paths. To reach the miraculous planet Ultima it described, you had to break the rules.

The legacy of these multi-sequential books lives on in digital interactive fiction (IF), which was among the first game genres made possible by computing. IF, which can be presented on the web, in standalone apps, and even in print, presents readers with choices that alter their path through a work. Jason Shiga’s Meanwhile (2010), a graphic novel boasting 3,856 possible readings, uses a print analogue to hypertext: pipes that extend from a sequence of panels off the edge of the page to create a kind of tabbed thumb index by which one can leap to other points in the book. Designed to emulate what comic book artist and theorist Scott McCloud calls an “infinite canvas,” Meanwhile also exists as an app in which all potential paths are available in an interface that scrolls in every direction.

Meanwhile

Interactive books come in other game-like forms, including Mad Libs, storytelling dice and decks, and magnetic poetry. Publishers and book artists have used the deck of cards as another playful model for the book that can be sequenced by the reader. John Cage’s work with indeterminacy in the 1960s might be included among such works; as would French author Marc Saporta’s Composition No. 1 (Éditions du Seuil, 1961), a box of 150 leaves printed on only one side that the reader is instructed to shuffle at the outset; and B. S. Johnson’s The Unfortunates (Panther Books, 1969), whose opening and closing quires enclose twenty-five sections that may be read in any order. This bracketing method, in which the story’s opening and closing are set, was used by Robert Coover for “Heart Suit,” a story in McSweeney’s Issue 16 (May 2005) printed on fifteen oversized heart-suited cards including a title card and a joker providing the tale’s introduction and conclusion. Artist Christian Marclay, whose work focuses on found and appropriated materials, published a deck of cards called Shuffle in 2007 that, in Cagean fashion, presents the reader with seventy-five images of musical notation in situ (as a decorative element on mugs, jackets, murals, and the like), which are meant to be shuffled to create a playable score.

Artist Carolee Schneeman’s ABC—We Print Anything—In the Cards (1977) is seminal in this regard. Consisting of 158 colour-coded cards in a blue cloth box, the work was intended as a score that could be variously interpreted by the reader. Including dream and diary excerpts on yellow cards, quotes by characters A, B, and C (based on Schneeman; her soon-to-be ex, Anthony; and her new lover Bruce) on blue cards, and comments from friends on pink cards, the book suggests that as a relationship ends, it can feel as if every possibility were predetermined, or “in the cards.” “We print anything,” perhaps the slogan of a print shop or tabloid, tells us that this ABC, far from rudimentary schoolbook, is for an adult audience, and that it holds nothing back, just as Schneeman kept little off limits in her body art and performance work. Black-and-white photographic cards intersperse images of her nude body, her domestic space, and erotic artwork as if to reinforce the fact that the book lays all her cards on the table.

What happens, though, when a book is boxed and unbound? Do we still recognise it as a book? Of course we do; the box acts like a familiar slipcase for a hardbound book. It presents a rectilinear volume that can be arrayed on a bookshelf, and it contains the pages or cards that come together in its content. Yet, while it looks like a codex from outside, the moment we open the box something changes. These pages can be “turned” in that they can be flipped over, creating two stacks of loose sheets facing one another. Is the space between them properly an “opening” as one finds in a codex or accordion book? Yes. And no. In an accordion or codex, the author and designer have conceived of the opening and the interplay between the facing pages. In an unbound book, that interplay will be different each time it is read, since we can shuffle and reorder them at will. If the cards or pages are not numbered, then the order is truly left up to the reader, and perhaps even the orientation—the page can now be rotated (though in some cases, this will render its text illegible without a mirror or Blake’s skills).

Some of the loveliest works to play with this potentiality are Swiss-German poet Dieter Roth’s (born Karl Dietrich Roth, 1930–1998) series titled simply Bok (Book) from the late 1950s and early 1960s. Born in Germany, Roth’s parents sent him to Switzerland in 1943 for the duration of the war (his family reunited there in 1946) and there he trained as a graphic designer, met concrete poet Eugen Gomringer, and began experimenting with visual poems and artists’ books. When he moved to Reykjavik in 1957, Roth created his own small press, forlag ed., and began to issue books in a variety of cut-paper formats. Famously playful with book form, his first publication, Kinderbuch (Children’s Book), originated as a gift for his friend Claus Bremer’s son and consisted of twenty-eight 32 x 32 cm pages letterpress-printed with red, yellow, black, and blue circles and squares in a variety of arrangements and sizes. The spiral-bound book was produced in an edition of one hundred, twenty-five of which also had die-cut shapes, which would become a technique of great interest to him. That playful spirit continues in the schlitzbücher (slot-books) he began work on in 1959. These collections of loose cardstock pages, each around fifteen inches square with a smaller central square of hand-cut slots varying in width and orientation, have an immediately cheeky quality. Rather than titling them, each bok was given a number or double-letter designation. Minimalist in aesthetic, they consist of ten to twenty-four leaves of cardstock in two or three colors (black and white, red and blue, red and green, blue and orange, and in one case, red, green, and blue) encased in a portfolio. When stacked and turned by the reader, they alternately reveal and conceal portions of the pages below, creating a variety of optical effects and transformations. The portfolio format, here as in Blake’s illuminated prints, reminds us that our definition of the book cannot rely on formal qualities alone—a book’s meaning arises through use and through the apparatus set up to shape our interpretation of it.

Because they are unbound, each leaf of the slot-books can be oriented four ways (not all are symmetrically centred) as well as flipped, offering eight possible orientations for each sheet, which in turn are influenced by the arrangement of the pages below. These interactive works play with our notions of the book by presenting us with a space that references text (that central cut-out area evoking a prose block with ample margins), but that only becomes legible through flipping—rather than moving our eye to scan these lines, we move the page to make meaning from it. Though we can examine and appreciate an individual sheet as a work of op art, we must, in fact, look through it for juxtaposition with the page below, much as a single page of text gathers significance through its place in a book’s sequence. One such recombinant book has been remediated by generative artist The55 into a visual simulation that allows us to layer the pages to our heart’s content, illuminating the extent and variety possible in the work, which must be activated by a reader to generate meaning, since, after all, the pages contain no text.

The Book by Amaranth Borsuk
$15.95 TISBN: 9780262535410344 pp. | 5 in x 7 in21 b&w illus.

Available from MIT Press