Virtual Reality Filmmakers Say Questions Outnumber Answers -- And That's Okay

On the eve of consumer virtual reality, with a slew of VR headsets based on common mobile phones going on sale this month and next, the buzz has turned away from gaming and toward movies and other media “experiences.” And there’s still a lot we don’t know about how those experiences should look.

People experimenting with VR movies, though, remain confident that they’re on the right track, and say the reason questions outnumber answers right now is because of how much is changing in VR.

For readers who haven’t tried on a virtual reality headset yet, it might help to understand what content creators are trying to do. While many game developers, the first people to support headsets like Facebook’s Oculus Rift, say they want players to make virtual worlds that are as believable as real ones, Jaunt CEO Jens Christensen said the aim of cinematic VR is slightly scaled back.

“Our goal is to achieve an emotional connection with a user, to forget about the technology completely,” Christensen said. “That’s the goal, to feel like you’re there and amazed by the beauty of the music or the nature that you happen to be in.”

In other words, they don’t need to fool you — just entertain you.

Still, Christensen’s goal is more or less in line with the VR content world’s favorite buzzword, “presence.” That’s shorthand for any sort of VR experience (game, film or otherwise) that convinces the user of something unreal. In Jaunt’s case, the challenge is to make the user feel the energy of a live Paul McCartney concert, even though they aren’t really onstage with Sir Paul.

A new short film called “Zero Point,” released by a different cinematic VR company called Condition One, briefly achieved presence for me in a seemingly pastoral scene of bison walking through a field. My head naturally followed the bison along their path, but I noticed they were turning to look back at the rest of their herd — so I turned, too, only to get a face full of curious beast.

“‘Oh! The bison is right there!'” Condition One CEO Danfung Dennis quoted another viewer as having said. “Everyone has that experience.”

In reality, of course, the animal was inspecting Dennis’s 3-D camera rig, which probably looked out of place in the middle of a field. But as a viewer, taking the place of that camera, I instinctively leaned away to try and get some distance from a thing that wasn’t there.

So, if it’s possible to get presence, what are the problems?

For starters, the resolution of the screens on mobile phones — the only devices supporting consumer VR at the moment — is lower than the film guys would like.

“The screen resolution is really perfect now if you hold it at arm’s length,” Christensen said of his Nexus 5, a premium Android phone. “When you bring it up to your eyes, with lenses, suddenly you need more.”

When phones are too close to your eyes, Dennis added, you get a “screen door effect” — meaning you may notice lines in the image that break the feeling of presence. Future devices need to have panels that are at least 1440p to fix the problem, he said.

"Zero Point" / Condition One

Another big aid to presence that you won’t find in the first round of cinematic VR content is positional tracking. What that means in layman’s terms: You can turn your head to see more of a scene happening all around you, but you can’t currently move your head or body to inspect different facets of a 3-D object. When the bison came up to inspect me in “Zero Point,” I leaned away out of instinct but my perceived distance from it stayed the same.

One source experimenting with video in virtual reality at a prominent company, who asked not to be named, said cinematic VR will never really click with audiences until positional tracking is possible. It only works now in computer-generated experiences such as Oculus’ Crescent Bay demo, which used an external camera to track users as they walked around in virtual rooms.

Christensen and Dennis said positional tracking in live-action video is possible, to an extent. Dennis speculated that the best way to achieve this is to use special depth-sensing cameras, similar to the Xbox’s Kinect, to collect data about how far away objects are from the various normal-camera lenses. This information could make the 3-D effect of a video more dynamic, changing as users move their heads around, though it still wouldn’t get a bison out of my face.

The last questions is a big one: Just what will filmmakers trained to shoot movies for 2-D screens be able to bring into VR?

Not a lot, as it turns out. In both Jaunt’s Paul McCartney demo and Condition One’s “Zero Point,” camera cuts are eschewed in favor of slow fading transitions. And for the most part, no matter where a camera is, it’s holding still.

“We’re so used to cuts now,” Christensen said of scene transitions. “When they first introduced it to cinema, cuts were a big deal. It took a while to figure out the whole language. Now people are comfortable with them, and that extends to VR, but not moving the camera when you’re not moving, yourself.”

That’s because the brain wants vision and motion to align; when there’s a mismatch, some people feel sick. Dennis said Condition One has learned a counterintuitive truth while shooting VR content, though: When the camera does need to move, slow movement may be the worst kind for motion sickness.

“In traditional filmmaking, you ramp the camera up to speed and ramp it slowly down to stop,” he said. “In VR, you want to go instantly from zero to 60. It minimizes the mismatch between your inner ear and what your mind is seeing.”

And some kinds of movement are just unworkable. Dennis said a scene in “Zero Point,” shot from the perspective of someone slowly riding down an escalator, had to be cut because the artificial altitude change made a lot of viewers sick.

“Moving down or rising at 45 degrees is very uncomfortable,” he said. “The tripod can be very limiting, but it’s the safest thing so far.”

Despite all this, content creators say the added immersion of virtual reality outweighs the current limitations of the technology. Jules Urbach, the CEO of Los Angeles-based rendering company OTOY, said VR is a “very easy sell” in Hollywood at the moment.

“VR is something that all six major studios and other content creators take very seriously,” Urbach said. “At this point, everyone kind of gets it. They remember Cinerama: Three screens were more immersive than one. Movie studios understand that greater immersion is a big deal.”

This article originally appeared on Recode.net.

Will you join us?

Our biggest supporters are our readers — and we’re so grateful to everyone who has made a contribution during our September campaign. We’re less than 1,000 contributions away from reaching our goal for the month, which in turn will allow us to say yes more often when our incredible journalists come to us with questions they want to answer and projects they want to pursue. Will you make a contribution before the month ends and support our policy coverage through 2024 and beyond?

One-Time Monthly Annual

$5/month

$10/month

$25/month

$50/month

Other

$

Yes, I'll give $5/month

Yes, I'll give $5/month

We accept credit card, Apple Pay, and Google Pay. You can also contribute via