A book about a girl coping with the loss of her beloved puppy is at the heart of an immersive virtual-reality experience from internationally renowned Taiwanese author-illustrator Jimmy Liao, whose 50-plus picture books include The Sound of Colors, The Blue Stone, and When the Moon Forgot, which have been translated into English and published by Little, Brown. Liao is also the illustrator of Joyce Dunbar’s The Monster Who Ate Darkness, Sean Taylor’s The World Champion of Staying Awake, and Jerry Spinelli’s I Can Be Anything. His books have been translated into a dozen languages, including English, French, German, Italian, and Spanish. Several of his titles, such as The Sound of Colors and A Fish That Smiled at Me, have been adapted into musical dramas and animation. In 2015, Liao founded Jimmy S.P.A. Plus to extend his works into the digital sphere; a collaboration with consumer electronics company HTC, also based in Taipei, soon followed.
With the application of HTC Vive technology, four scenes from Liao’s latest picture book, All of My World Is You, are now rendered in a room-scale panoramic virtual world that offers precision navigation, interaction, and communication. The four paper-to-3D modules, created by Taipei-based Next Animation Studio, deliver lessons on love and loss through interactions with different characters: the gardener nurturing a bizarre plant, the TV-addicted monster, the piano mover who used to be a concert pianist, and the heartbroken lady in green. Launched at the 2016 Taipei International Book Fair and exhibited at the subsequent Bologna and Frankfurt fairs, the VR modules have become more popular with each event. At the recently concluded 2017 Taipei Book Fair, there was a waiting list to experience the virtual world of Jimmy.
Looking back, business development manager Jeannie Yong of Jimmy S.P.A. Plus said that the biggest challenge in creating the project “was in bringing the pages and characters ‘alive’ for the VR experience. Even the production house had to experiment with it since VR is totally unlike traditional animation. The scripting had to incorporate game and probability theories in order to make the player-character interactions fun, real, and reactive. It was a whole different game.”
The make-believe world, Yong added, “has to be smooth and seamless, with the player’s reaction and responses determining the depth and length of the interaction. Creating a virtual world in this case means taking the illustrations in the book way beyond the printed pages and book margins. We have to develop the full-scale 360-degree environment.”
The next challenge for the overall VR world is gaining wider public acceptance and adoption. In Taipei, the opening of HTC’s first VR/MR (mixed reality) arcade—HTC Viveland—provides the space and opportunities for public VR experimentation. Across the straits in mainland China, companies big and small are jumping onto the VR technology/hardware bandwagon, while VR arcades are springing up in the main cities, prompting analysts to predict that the Chinese VR market will be worth $8.5 billion by 2020. HTC already has two Vive VR Café outlets in China, and is looking to add hundreds more. For now, the level of VR experimentation and commercialization in China and Northeast Asia has far exceeded that in the U.S. and elsewhere.
Back in Taiwan, author-illustrator Liao has a permanent VR staging area in Yilan County, his birthplace, some 25 miles southeast of Taipei. There, one will also find art installations of characters from three of his books—Turn Left, Turn Right; The Starry, Starry Night; and The Sound of Colors—at the train station, bus terminal, and public spaces.
Presently, the VR experience is very much a single-player immersion. “Future development will see the technology allowing one player to ‘see’ and ‘interact’ with another player or players, making a multi-player environment possible,” added Yong, pointing out that “current VR experience is limited to arcades, especially in the big cities due to living space constraints and costly hardware purchases involving headsets and heavy-duty computers with graphics cards.” But with major players—including Facebook, Google, Microsoft, and Sony—busy working on the next VR technology, the hardware cost will go down amid improved research and development and stiffer competition.
Another challenge is monetization. Yong is looking at placing the modules on the HTC Viveport VR App Store, which is the world’s first VR content subscription platform. “Even as we figure out the best pricing scales, online portals, and further adoptions, we are looking into transforming another Jimmy title into VR,” said Yong, who is eyeing the Greater China territories for promotion and sales.
Many illustrators are keen to turn their works into VR modules and have approached Yong for advice. She tells them: “Find the right production house. Scripting takes time, and there is going to be a one-step forward, two-steps back kind of progress. Selecting the right content is crucial.”
It took Next Animation and Jimmy S.P.A. Plus about 10 months from start to finish to construct the four modules. “And when we look at the finished product,” Yong said, “it is undeniable that VR is a great technology and fit for extending the value and content of a picture book into an entirely new, exciting, and immersive space.”
