Children’s literature has always been a genre curious to experiment and play with media. Just think of pop-up-books and how the Alice in Wonderland adaptation Alice for the iPad (2010) by Atomic Antelope was one of the first apps to explore the features of the iPad. Since the introduction of the iPad in 2010, several attempts have been made to explore meaningful alliances between the print book and the digital device using augmented reality technology; creating hybrid experiences combining the traditional medium for children’s literature and its newest carrier. Such experiences depend on the user installing an app on her digital device and pointing its camera at the pages of a book. The device reads or decodes the data on the paper page and activates and displays content on the screen. The user’s physical, multi-sensory and cognitive engagement in such experiences is also one of complex character as she is managing two technologies at the same time. 

On the one hand, the book or codex technology which include turning the pages and navigating in specific ways, and on the other hand, the digital device and its specific interface navigation. In this way, and as is the defining nature of augmented reality, the book that exists in the user’s real-world environment is enhanced by computer-generated sensory information thus playing with the user’s perception. Off-hand, the effort to create this union seems slightly paradoxical if we consider the fact that the iPad was conceived and designed specifically as the unification of the book and the computer. So, what can actually come of this persistent Sisyphean task of making the print book and the tablet computer work together? 

This article will dig into this question by taking you through some remarkable international examples of literary AR book projects for children, their development over the past decade and the experiences they can produce. It will then shift from an international to a more local, Danish perspective and explore the potential of AR books for children for supporting reading motivation. In Denmark, a new partnership saw the light of day in 2021 in the company Smart Books. The company consists of the popular YouTuber and influencer Rasmus Kolbe, best known under his old boy scout name Lakserytteren (directly translated: the salmon rider) and Søren Jønsson, who is a successful and experienced producer of games for children. Smart Books deliver an augmented reality ‘smart book’ concept, where the reader chooses the path through the paper book’s narrative, interacting with both a book and digital content on a smartphone or tablet, and in this way gains an interactive reading experience. While this strategy is new in a Danish context, the venture also stands on the shoulders of a line of previous AR book projects.

Literary “AR + book” projects for children

When looking over the last decade’s projects that combine AR and paper books aimed at children, it is clear that this media interplay has gained a stronger footing in non-fiction and educational publications than in more literary, narrative projects. Generally, in these latter projects there has been a development from early projects that mainly ‘digitize’ the content of paper books, such as 3D animation of characters in picture books (e.g., Resin’s Two Left Feet, 2013), without asking for the user’s engagement to any significant degree, to newer projects that play with the potential of the augmented digital environment more fully and call for the user’s engagement via interactive game elements (such as Books & Magic’s The Little Mermaid, 2016). However, in some of the latter projects the print book’s materiality and role is in turn neglected. The crux of the matter seems to be to find a balance between the media where one is not a gimmicky appendix to the other. 

Resin’s Two Left Feet

Books & Magic’s The Little Mermaid

If we look to projects that can be characterised as literary in the sense that storytelling and an aesthetic experience are at the forefront of the works, projects produced  by the now hibernating American multi-platform storytelling company Moonbot Studios stand out. These works include The Fantastic Flying Books of Mr. Morris Lessmore and The Numberlys. Both works are aimed at children, The Numberlys at young children (3-7 years old) and Lessmore at slightly older children, yet a target audience is not mentioned anywhere. They both exist in several media at the same time constituting cluster works (Mygind 2017): as picturebooks and AR apps that can be used in conjunction with the picturebooks, as standalone interactive apps and animation films. In the changeable, fleeting world of apps, these works are already old (the books, the apps and films came out in the period 2011-2014) and are not available for purchase anymore, but this does not mean that they are not worth mentioning here. On the contrary, these works draw closer to a balance and a mutual relationship between the print and digital media than many other works. 

Moonbot Studios’ The Fantastic Flying Books of Mr. Morris Lessmore, teaser for all versions

Moonbot Studios Imag-n-o-tron: Numberlys edition

The picturebook The Numberlys takes full advantage of the book medium by playing with the reader’s ingrained expectation to the book by using the oblong format and mixing the reading directions of the book. The book is bound in the short, left side, which would normally mean that the book is read horizontally, but already from the title page the expectation is denied as this page must be read vertically and the book must, therefore, be turned. The title is one large image that spreads from top to bottom, with the five little main characters marching across the page at the bottom. This vertical reading direction enhances the impression of the vast, oppressive world of all-pervading numbers that the characters inhabit. This is a characteristic of the picturebook that it takes advantage of the book medium, its materiality and reading conventions and plays with them to convey meaning. In relation to the hybrid AR book experience, it is noteworthy to point out how the interactivity between the reader and the medium becomes part of the way meaning is conveyed in this universe and therefore not something that is reserved for the digital component.  

In relation to the rest of the cluster work that The Numberlys compose through its many independent media versions, the AR app IMAG-N-O-TRON: Numberlys Edition is the only one that cannot stand alone. Since the app is dependent on the picturebook to activate its content, it can be characterised as an intracompositional transmedia phenomenon (Dena 2009), meaning that the ‘AR app + book’ composition is transmedial in itself, and the app is not a self-contained narrative unit. When opening the AR app, the reader is transformed into and staged as a player, collector and detective, using the digital device as a magnifying glass through which to explore and investigate the picturebook. The AR app encourages the player to scan the book for objects, which,  when located on the paper page via the camera, will turn into animated objects on the screen and be stored in the apps interface. When the objects from the book have been collected, the player can build new, fun constructions in the digital space and practice constructing letters and numbers. In this way, the app encourages the player to perform creative, educating tasks that mimic what the fictional characters do in the picturebook, thereby extending what we might call the core values or message from the picturebook to the digital environment via the AR technology.  

In relation to the aforementioned balance between the media in AR book compositions, it is noteworthy how, on the one hand, The Numberlys picturebook composes a self-contained narrative entity, while, on the other hand, the ‘AR app + book’ composition actually works independently as well. Of course, the reader/player will gain a deeper, more informed experience if she reads the story in picturebook, but it is not a prerequisite to engage with the part of the Numberlys universe that is available in the AR book composition as they offer two distinct kinds of engagement: engagement in a narrative and engagement in playful activities.

Disrupting reading and media cultural hierarchies

Moonbot Studios’ few productions were, and still are, innovative and remarkable examples of AR + book compositions and, on the whole, of experiments with multi-platform, transmedia storytelling, however, the venture did not continue and did not set a precedent for subsequent international AR books for children. If we look to the recent Danish Smart Book concept, this endeavour is targeted at older children, specifically children from 9-13 years old, and here we find yet another approach to the composition of the AR book. 

The Smart Book concept consists of a series of paperback books with individual accompanying AR apps. Currently, three books have been published in Danish and two in English are forthcoming. The series takes place in a fantasy universe of wizards and magic where the reader assumes the role as the protagonist “you”, the First Student of the Firemaster. Just like in the so-called gamebook series for children from the 80-90s Choose You Own Adventure, you navigate the book by reading short numbered chapters that present you with a choice and, depending on the choice, directs you to a new chapter. Some chapters also present the reader with challenges and puzzles, often small maths related puzzles, that must be solved to move on. These obstacles are presented in the book via simple illustrations and text but must be met and solved in the AR app that, when accomplished, will direct the reader to a new chapter in the book. The content of the puzzles is most often not related to the narrative and therefore not narratively motivated. Instead, they offer different kinds of cognitive engagement.       

An interesting aspect of this AR book composition is that the traditional way of reading a book is turned into something else via the non-linear and unknowable reading path. We might say that the book and the AR app create both a material and fictional space in which the reader moves back and forth, yet with a feeling of moving forward without knowing when the journey will end. Normally the reader of a book can see, feel and count the number of pages read and the number of pages remaining in the book. This conventional way of navigating a narrative in a book is suspended and disrupted both by integrating the AR app and through the non-linear structure resulting, paradoxically, in both a higher degree of agency assigned to the reader and a higher degree of obscurity or mystery for the reader.

In their public presentations of the Smart Book concept, the two owners of the company, Søren Jønsson and Rasmus Kolbe/Lakserytteren, associate their project and their motivation behind it with the widespread concerns about the decrease in reading among children. A Danish study of children’s reading habits conclude that children’s joy of reading literature decreases with age and that the drop happens between 5th and 6th grade (Hansen et al. 2017). This situation is the point of departure for the Smart Book project, and the owners’ stated mission is to reignite children and young people’s joy for the written word, and showing them the way to the AR books especially through Lakserytteren’s media channels, such as YouTube, TikTok and Instagram.

The Smart Book series adapts the known “choose your own adventure” form to the AR book, hereby disturbing existing notions of what it means to read literature and proposes a new way of reading. These hybrid works of fiction combine print and digital media in a mutually dependent fashion that challenges prevalent public debates about onscreen reading. In these debates a notion of a media hierarchy is formed. A hierarchy that contrasts print media as the authentic, educational way of reading with digital reading (both visual and audio) as the inferior way of reading. Transmedia AR book projects have an intrinsic potential of being able to break down this hierarchy and offer new multi-sensory, transmedia experiences that support different reading styles. 

References

Dena, C. 2009. Transmedia Practice. Theorising the Practice of Expressing a Fictional World across Distinct Media and Environments. PhD dissertation.

Hansen, S. R. et al (2017): Børns læsning 2017: En kvantitativ undersøgelse af børns læse- og medievaner i fritiden, Læremiddel.dk: Tænketanken Fremtidens Biblioteker.

Mygind, S., 2017. “A Chinese cluster: Danish-born digital comic as source for transmedia design and innovation” in Ensslin, A. et al (eds.): Small Screen Fictions. Vashon Island, Washington: Paradoxa