Where Is The Bird? – A Story of Augmenting Reality

Where Is The Bird? is the first augmented reality storybook to promote British Sign Language as a language for deaf and hearing children alike. Combining animated Augmented Reality (AR), video and audio with traditional print media, it endeavours to create an immersive multimedia storybook. By turning the pages the reader can find and ‘awaken’ magical illustrations that pop-up out of the pages as colourful 3D animations before being paired with video demonstrations of BSL sign. 

 It all started when two friends were having a conversation about how the balance of readers consumption of media has changed. Less time is being spent learning from books and instead people watch documentaries, listen to podcasts, download apps or research through the plethora of online platforms for information on their favourite subjects. This conversation led to many ideas, including  the partnership of VIKA Books, a female-led publishing team, merging creative technology with book arts to conceive new immersive books. 

While some see screen usage as the enemy of books, VIKA realised there could be an opportunity to combine the best visceral aspects of the physical book with the immense creative possibilities of digital. This led us to begin thinking about how books could link with smartphones to enable intuitive interplay between the printed page and content such as videos, quizzes, interviews or even augmented reality, (AR). We wanted to give readers the opportunity to immerse themselves in their chosen subject however and wherever they wanted – to be able to watch videos, listen to audio and even share stories with friends yet simultaneously curl up with a book at home and stow it on a bookshelf filled with beautiful titles. And so we began re-imagining the book as the gateway to a whole new way of reading. 

The commercial gap for ‘Where is the Bird?’ was identified after a laborious personal experience searching for British Sign Language (BSL) teaching aids. Teaching materials for BSL are surprisingly limited and not always entirely clear as sign language has a dimensionality and depth, which cannot be captured across the pages of a book. Deaf children can often find books uninspiring as the written word is a static, sonic, linear language – separate to the visual, dynamic language of BSL. We were inspired to fill this gap in the market and enrich the learning experience by using digital materials as a solution. From previous projects and general market research, we noticed how imperative it is to consider why and how we might use technology with each book – it needs to marry with the content and hold purpose rather than arbitrarily added as a gimmick. However, it seemed to us that as a uniquely visual language, BSL would be well represented by AR, video, illustration and touch. 

Development

In October 2018, VIKA won InnovateUK’s Audience of the Future Award to prototype a series of products demonstrating the multi-media AR interface used in Where is the Bird? VIKA worked in partnership with BDH, an established team that creates award-winning content for digital media, alongside partners from within the deaf community including VS1 Productions, the Deaf Studies Trust and Elmfield School for the Deaf. 

 The project’s first phase involved a human-centred study, which focussed on discovering market research knowledge, product design knowledge through prototyping and user evaluation. We created many versions of the book and our results were logged, evaluated and used to inform the next series of design iterations. This process was repeated many times with parents, pre-schoolers and members of the deaf community which gave us invaluable insight and guidance and provided our first clue for creating a commercially innovative product: always, always think about the audience’s needs and wants. Without this user testing, assumptions would have guided us incorrectly and we would never have been unable to achieve this successfully. 

By the end of the process, Baby BSL had become an interactive system of learning. It can be read as a storybook without using the digital additions or it can be used alongside the AR and the videos; alternatively extra information can be accessed through the app so readers can pick and choose how they want to engage with the world of Baby BSL at various points in the daily routine. It is designed with a soft-touch approach to introduce immersive technologies in a gentle, ethical and unobtrusive way to young families. We believe Where Is The Bird? is a book, which is enhanced by the power of AR and accomplishes its purpose more successfully because of the use of technology.

In April 2019, VIKA received the South West Creative Technology Network (SWCTN) Immersion Prototype Award to produce a prototype Baby BSL buggy book for distribution and test the market for commercial scalability. This let us invest into a range of communications strategies alongside three complementary agencies: Oggadoon, Ladbury PR and MSQ Partners, as well as researching relationships with bookshops, distributors, partners, and community hubs in order to get Where is the Bird? to as many families across the South West as possible. 

VIKA’s Challenges

Many people still don’t know what augmented reality is or the potential it has. For new innovations, such as AR, success is dependent on both the end user and the marketplace dramatically changing their past behaviour in exchange for new benefits. Many of our books will include tech as an integral part of the experience and so our counter to this problem is to try and guide users through that learning process in a clear and simple manner.

Additionally, sourcing funding to support risky, early-stage innovation within a fledgling market is hard work as a venture such as this requires developers, animators and sound technicians which is a costly effort to pursue, with hefty upfront costs.

The Next Step

Whilst working on Where is the Bird? our focus was mainly on driving the storytelling and experience and less on making the AR models as sophisticated as we could. With our newfound knowledge of this medium however, we now know how to improve integration onto the page and have a big imagination for what this can achieve next time. Our new focus is to provide more interactive features for the audience when engaging with a book – and making the book a portal itself into the world of Baby BSL. 

Natural progression for AR is to replace hand-held devices with wearable ones, like glasses or a pair of lenses. Whether that will coincide with digital paper we’re not sure, it depends on the development of holographic projection for 3D animation to takeover the page. It seems that people will expect any given surface to have information available in the next generation of interfaces and it’s the reason people are excited about AR.

Where is the Bird? has been an interesting project and has helped us concrete the motivations of the company. This opportunity allowed us to test out new partnerships and start thinking about our possibilities for the future.

Facebook

Twitter

Instagram