The Prospect of VR and AR as Storytelling Platforms: A Critical Assessment

A Critical Assessment of VR and AR as Storytelling Platforms within the Entertainment Industry

On May 19^, 2016 I gave an invited key note address in Auckland kicking off Magnify, the first South Pacific Conference on Augmented Reality and Virtual Reality as part of New Zealand’s Tech Week. Sponsors for the event included Air New Zealand and Sky City Entertainment.  The conference attracted over 1,500 attendees that packed the house for both of the two days. Dignitaries from the New Zealand Government were in attendance along with a full slate of exceptional speakers.

Specifically, I was asked to give an overview of the impact that virtual and augmented reality is currently having on the entertainment industry and in particular their place in cinematic storytelling. The organizers asked me address the new display technologies from my unique perspective having been an innovator in digital visual effects and display technologies in Hollywood over the past three decades. Below is a synopsis of my presentation.

VR and AR as Storytelling Platforms

There is little question that there are many viable niche applications for VR that will undoubtedly find a consumer base and that ultimately will become successful revenue generators. However, as a true storytelling medium I believe that many of the strongest proponents within the entertainment and consumer electronics industries as well as many high profile investors in Silicon Valley have been living in a bubble where like minded entrepreneurs, engineers and creative professionals have tended to feed off of their mutual hype without much regard to the consumer and the potential of a mass market.

Indeed, it reflects a recurring syndrome that often accompanies new technologies, evoking visions of those proverbial lemmings, following each other... jumping off a cliff en masse.  In many ways, VR in 2016 is reminiscent of the feeding frenzy that we saw at the 2010 and even the 2011 Consumer Electronics Show where, despite the absence of a sufficient catalog of compelling 3D entertainment content, every display manufacturer was keen to show off their latest and greatest 3DTV consumer technology. At CES, 3D and talk of 3D was everywhere.

I always feel that when the hype surrounding new display technology becomes blinding without a clear avenue toward monetization, it’s important to take a step back to re-assess. If you can't make money with the new technology, then it becomes little more that an interesting academic exercise.  My career started out in academia and I have no intention of returning to those days of esoteric interests at the expense of more lucrative entrepreneurial pursuits. People heavily invested in VR for entertainment purposes have to ask themselves, is there a genuine interest in complete immersion from a wide enough demographic spectrum to generate the kind of revenue being projected by Goldman Sacks and others?  Perhaps the millennial generation will find VR compelling for entertainment purposes but so far I see it as little more than a curiosity among most people outside of gaming. In fact, I’ve yet to see a fully immersive VR entertainment title, billed as storytelling that reflects little more than a theme park ride experience or interactive game experience masquerading as story.  All great fun and interesting experiences… but they are definitely not narrative storytelling. The biggest question is whether people want to be told a story in a passive manner or if they want to be fully immersed in a story. The jury is still out, but based on my experience in both colorization and 3D conversion industries, the widest demographic would rather be told a good story passively with compelling and well directed visuals.

I’m not alone in this view. When Ang Lee recently screened his film, “Billy Lynn’s Long Halftime Walk” in high frame rate (120 frames per second), high dynamic range, ultra high definition and 3D using laser projectors he was asked what he thought of virtual reality, his response was telling… “my virtual reality is better than your virtual reality, let me move the camera.” I believe that sentiment is common among established directors who have even bothered to consider the new medium. When asked about virtual reality at the Sundance Film festival last year, both George Lucas and Robert Redford agreed that the 30 virtual reality ‘film’ titles on display at the Festival were interesting... they had all the elements of cinema… actors, a script, visual effects, etc… but there was no story. In a similar but more diplomatic vein, Spielberg said VR will "take hold in a profound way" but cautioned. "...it is dangerous... because it gives the viewer a lot of latitude not to take direction from the storytellers but make their own choices of where to look," he said in an interview. "I just hope it doesn't forget the story when it starts enveloping us in a world that we can see all around us and make our own choices of what to look at."

That said, I see current experiments and commercial efforts to make VR into an entertainment medium a precursor to Augmented Reality which I believe will be come ubiquitous in 3 to 5 years. Almost anything you want to do in VR can, or eventually will be accomplished in AR.  The difference however is that in AR you are maintaining a presence in your own socially interactive reality without having your senses kidnapped within a forced sense of isolation as in VR.  Yes, AR is a somewhat different experience than VR but one that I believe will ultimately be more acceptable and more compelling to a mass audience. Another relevant lesson we learned from 3D... it's not going away even though the home entertainment market largely failed. It remains a significant revenue generator theatrically for the studios but it’s been largely confined to the genre of super heroes, animation and fantasy.  Similarly, I believe VR will find its place in entertainment but more confined to gaming, entertaining simulations, documentaries and theme park-like rides in your head.

The Greatest Challenges to VR and AR

The greatest challenges to innovations in VR and AR are consumer adoption, consumer adoption and consumer adoption, and 100% related to consumer adoption… compelling content that people will actually spend money to experience.  VR for gaming applications and related cinematics is a given and the choice in headsets is narrowing. For AR, compelling content is being produced  and anyone with an IOS or Android mobile platform can experience AR today.  A significant advance for both AR and VR are untethered optics that have a relatively small form factor.

Critical Timeline  for VR Storytelling to Attract a Mass Market

I think it’s fair to say that if there is no compelling entertainment content outside of gaming at the 2017 Consumer Electronics Show in Las Vegas, VR as an entertainment platform will have started a dive down the hype curve from its peak of inflated expectations where I consider it is today, toward the trough of disillusionment. If the situation persists, it will be at the trough of disillusionment or optimistically, on the upslope to enlightenment by CES 2018.  Keep in mind, that while this is happening in VR, I believe AR will begin to generate significant traction in entertainment sectors; particularly if Hololens and Magic Leap come through with affordable optics and truly compelling AR content.  I’m already seeing examples of compelling content from Hololens despite the limited field of view of the current headset, but to date all I’ve seen from Magic Leap are some great visual effects hype demos created by WETA. Bottom line for Magic Leap: If they hope to be successful, they must come through with affordable yet innovative optics as well as content that blows away Hololens. Raising $1.4 Billion USD to create their unique brand of optics renders Magic Leap so high profile that they've made themselves a target and whether they are successful or a failure it will become epic.  They will become either a boon to AR or a black eye on the nascent industry and market.  There’s lots of pressure on them. I suspect they might be building a library of content designed for their evolving platform before an official launch but that’s only speculation and they have a great content partner in WETA along with investor Richard Taylor. There is also competition from Metavision, the Osterhout Design Group and several other innovators in AR optics that might achieve significant traction in the relative near term.  

While I believe consumer grade AR optics will take another year or two to emerge, IOS and Android mobile platforms are already compatible with AR experiences and with 3D sensor and motion tracking technology coming to your mobile devices in the very near future, it’s a good bet that AR will begin a path toward ubiquity for consumers.

Disney funded Jaunt's CEO just announced he was leaving the cinematic VR startup.  Is this a sign of things to come? VR cinematic content professionals have to start thinking mass market monetization or they might be destined to become a footnote in the emergence of this new platform. There will be those who will disagree with this view. Indeed, for some Hollywood professionals and Silicon Valley investors, VR as a storytelling medium has taken on an almost religious ferver, but like any religion, their strength of commitment is based on faith rather than substance... and therein rests the problem.