"It's been a revelation": why Simon Mayor and Hilary James's new book uses AR

"It's been a revelation": why Simon Mayor and Hilary James's new book uses AR

Last Friday saw the annual FutureBook of the Year Award go to The Ghostkeeper's Journal and Field Guide, "the world's first augmented reality novel" written and produced by Japhet Asher, digital director of Carlton Books.

So are we about to see a rush of AR-enhanced titles hit the market? AR has already been used to pimp several educational textbooks and apps, but other areas of the publishing trade have so far found it trickier to establish a worthwhile use-case.

World-renowned mandolin player and BBC presenter Simon Mayor and musician, artist and illustrator Hilary James would beg to differ. The pair believe that their new release, Of Death and a Banana Skin - a collection of personal stories, poetry and art - is a showcase of exactly what AR can bring to a book. Point a phone or tablet at the pages and they layer music, animation and poetry readings on top of the text.

However, scepticism about the true value of AR to the industry is still rife. So I asked Hilary James some bracing questions about their experience so far.

Congratulations on the book! What sort of enhancements does AR bring to the text?

AR brings exciting multi-media possibilities for writer/performers like ourselves: recordings of music; readings of the poems; sequences of still photography;  and, because the illustrations were all done directly onto an iPad, video playback of the illustrative process. While the book stands alone as an anthology of poetry and prose with colour illustrations, the inclusion of AR means that for the first time we can bring all aspects of our artistic endeavours together in one project.

Why did you decide to incorporate AR into the book? Is it a PR gimmick or do you think it will really meet a true target audience demand?

Throughout history, any new technology has always been open to the accusation of being a gimmick, but it’s the content on which it will be judged. We are primarily musicians; but for Simon the written and spoken word, and for me the visual arts, are equally exciting and important. It’s been a revelation to find we can incorporate everything. The main target audience will likely be aware of this and would expect, or at least not be too surprised, that we chose to use AR in the way that we have. 

What’s the reaction been so far - and how many books have you sold?

While our music concerts have always been littered with humorous anecdotes, for the past three years Simon has also been reciting a couple of his poems - to much audience encouragement. In addition, we recently made a short film, a collection of the AR clips, which we are showing at the start of all concerts following a short introduction from the two of us.

Although AR has been around for several years it’s still something new and surprising to a lot of people. It’s been fascinating to watch from the back of the auditorium; audiences have been encouragingly enthusiastic and responsive with unexpected bursts of applause and, to our surprise, laughter (in the right places!).  In addition to our recordings and live performance, we have successfully established ourselves over many years in the rather niche field of mandolin tuition and repertoire (Simon Mayor has six books which sell steadily through music shops). 

Of Death And A Banana Skin is obviously very different, and we are still in the early days of promotion, feeling our way into the poetry market. Because Acoustics Publishing is our own imprint, we don’t have to satisfy a publisher’s timescale with early returns; we have the luxury of being able to explore the market over a longer period, giving both the profile of the book and understanding of AR time to develop. Initial media reaction, from both print and radio, has been hugely positive both for content and presentation with reviews so far in the music press and interviews on BBC Radio 3 ‘In Tune’ and BBC local radio stations. 

What were the greatest challenges in incorporating the AR?

When we first considered incorporating the technology we were a little frustrated because it seemed to have good days and bad days. We were intrigued by it and saw its potential but it seemed to work well mainly in daylight, and was unpredictable in artificial light. We decided to go ahead with it anyway in the hope and expectation that it would be improved by the developers. After publication of our book the app in question was bought by Hewlett Packard and has improved enormously. It now triggers easily in artificial light and the technology feels much more solid. Simon now has a website dedicated to the book with video demonstrations of the AR and how access the app.

What would you do differently next time?

It’s been a fascinating learning process, a real exploration into new territory. While most of the drawings work perfectly as AR triggers, we have had problems with a couple of the more impressionistic images with insufficiently strongly defined lines. We were unaware of this when we went to print and have had to work around it. I've already started on another book, a collection of songs and stories for children, so this time I’ll be drawing with AR in mind. We’ve been showing parts of the project to children and will be trying it out during a tour of schools in Cambridgeshire next spring.

Was it hard to get bookshops to stock something including AR?

This is a question we can't answer yet as our sales are mostly through Amazon and at live concerts. Since the book stands alone without AR we can't imagine it would make any difference.

What other AR books have you yourselves enjoyed as readers?

AR has been used primarily in education projects to date, and we are unaware of any widespread use in the general book market. The problem for the big publishing houses would be that any AR content such as music, photography or videography would have to be commissioned or licensed with the associated additional costs. We are in an advantageous position in that all the AR content is our own intellectual property, and so free for us to use apart from the time involved setting it up.