The future of interactive cinematic VR is coming, and fast

AUSTIN, Texas — I’m staring out over the edge of a cliff, the great rushing Colorado River visible far below. The vista in front of me is stunning, with Southwestern hues of orange and red bursting from the rock faces. But it’s almost totally still. There’s no movement at all.

Although it’s beautiful, crystal clear, and obviously live-action video, I’m confused by that stillness. Then I tilt my head to look down and far to my right. There, just in time, I see two sprinting men leap off the edge of the cliff and soar for a few seconds before their parachutes open.

Had I not turned my head, I’d have missed them. But that was the point. The Oculus VR headset I had strapped on recognized the movement, and now I was looking at what I would have seen if I had actually been physically looking down into that Utah canyon: the two BASE jumpers disappearing into the gorge.

This was a demo of Jaunt VR’s technology — an end-to-end system now available to VR content producers that starts with a 360-degree stereoscopic video camera; continues through video encoding, editing, and post production; and ends with live-action footage filling 360 degrees of a user’s field of vision. Palo Alto, California-based Jaunt is now distributing content for Oculus, Samsung Gear VR, Mac, PC, and a number of cardboard VR systems.

Above: Jaunt VR’s 360-degree stereoscopic camera.

Image Credit: Jaunt VR

Virtual reality systems are among the most exciting technologies on the horizon right now. With Oculus VR, Sony’s Morpheus, Microsoft’s Hololens, Magic Leap, and Samsung Gear VR, to name a few, storytellers are facing a dizzying array of platforms to choose from, even as consumers will soon be selecting their system(s) of choice. That means we’re on the edge of an all-new kind of storytelling, and the VR platform companies and content creators are in the earliest stages of trying to figure out how to come up with the kinds of compelling stories that will do the technology justice.

Filmmakers’ excitement about producing content for VR was on full display at the Sundance Film Festival in January. There, attendees were treated to a number of the latest VR cinematic experiences.

Still, whether it’s live-action VR, computer-generated VR, or a combination of the two, we are, as Wolfgang Bergmann, CEO of the German TV network ARTE Deutschland,  put it at South by Southwest (SXSW) on Saturday, “at the beginning of the beginning of the beginning [in VR storytelling]. It’s like silent movie part two.”

‘Live and Let Die’

I had not known what to expect when the Jaunt demo began, but I got it in seconds. When the video switched from river canyons to live-action footage of Paul McCartney playing “Live and Let Die” during the last-ever concert at San Francisco’s Candlestick Park, I was looking straight at the ex-Beatle. But by turning my head towards the tens of thousands of people on hand for the performance, I saw exactly what I would have seen had I been standing in that spot during the concert.

This is groundbreaking stuff, and it explains why top-tier venture capital firms like Redpoint Ventures, Highland Capital Partners, and Google Ventures have put $34 million into the company. The quality of the imagery is just about perfect, and the presentation instantly makes perfect sense. And, more to the point, it opens up a vast range of storytelling opportunities that have never before been possible. And Jaunt is home to some of that, putting out a slow stream of new VR experiences available to anyone using a device that can play it back.

At SXSW here this week, Jaunt was one of many VR companies on hand to showcase their latest technology. Jaunt, which has been producing publicly available content for several months, was showing this Utah cliff-diving footage, made in collaboration with the North Face, for the first time.

As the demo proceeded I tried to imagine the possibilities the technology was enabling.

Everything I’d seen of current-generation virtual reality tech was a presentation of computer-generated imagery. Stunning in its own right, the CG virtual reality had few storytelling boundaries. But you could always tell it was CG.

Here, with Jaunt, I was looking at live-action footage that felt as immersive and realistic as anything I’d seen in an IMAX theater.

Content creators lining up

That’s why Jaunt has content creators are lining up to work with it. The technology is a natural for musicians, sports, extreme sports, interviews, documentaries, and pretty much anything where someone would want to watch and immerse themselves.

It’s also the basis for a mixing of live-action and CG. In one Jaunt demo, I watched as a CG spaceship landed in an alley and an alien emerged, only to DJ electronica as a carful of live-action police screeched to the scene and pulled their guns. It was comical, and hinted at an infinite number of story possibilities.

It also made me think about what the panelists at a SXSW talk titled “Trailblazers in VR Domain” had told an audience about ways to bring functional interactivity to VR stories.

Above: Oculus head of worldwide studios Jason Rubin.

Image Credit: Daniel Terdiman/VentureBeat

So far, no one has managed to do much of that. As Jason Rubin, the head of worldwide studios for Oculus VR, put it, “Pre-rendered CG or film, that’s mostly story, and not interactive. When we’ve tried to combine (the two) it hasn’t worked that well.”

Rubin made it clear, however, that interactivity is coming — in some ways, the killer app for VR. “We’ll get to the Uber moment in VR,” he said, referring to the way the ride-hailing service brought a rich set of new uses to smartphones. “We can’t start with Uber. We’re starting with entertainment.”

One of the reasons Rubin is high on the interactive potential of VR is that the technology adds positional tracking, awareness and “agency,” the user’s ability to control the camera. “Now there’s a need for real-time stories,” he said. “You’re in the film in a way that’s never happened in film … because you’re actually there.”

If content creators can figure out a way to get the VR story right, Rubin argued, “it’s going to be incredibly compelling.”

One of the things that’s been missing so far in live-action VR is that the camera technology hasn’t had positional tracking, Rubin said. But with companies like Jaunt, that dynamic could be changing, and quickly.

For now, we’re still in the early stages, but as Rubin put it, “everything will be accelerated. … There will be a period of time when we’re figuring out how to create Academy Award winning … content in VR.”

An example of what he’d like to see in interactive VR storytelling is a game called Elite: Dangerous. In that game, as a VR headset-wearing user looks around the cockpit of a spaceship, he or she can focus on something and the game zooms in on it.

Choose your own adventure

Jaunt’s CTO and cofounder, Arthur van Hoff, said it’s possible to imagine using Jaunt’s technology to create stories where users can focus on one actor and follow that person’s story. A kind of VR Choose Your Own Adventure.

Or, van Hoff said, you might be able to push a button and see a different camera angle.

There are other ways to capture 360-degree live-action footage and build them into VR content, of course. But Jaunt believes that its purpose-built camera outshines what’s currently available — a mish-mash of GoPro cameras or others that were not built for VR.

But VR is hot right now, and it’s inevitable that there is going to be a rush of new companies, technologies, and investors looking to cash in. Jaunt hopes its head-start on creating its camera and production pipeline will set it apart, and help it be the platform of choice for the interactive content Rubin is clamoring for.

“Everybody wants to do VR at the moment,” van Hoff said. “Our our timing couldn’t have been better. … We started two years ago, ahead of most people. Today it’s all about the hardware — the headsets — and the frame rate. A year from now, that won’t be the issue. It will be all about content, and we’ll be perfectly positioned to” execute.