:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/63711456/585208372.0.jpg)
From the sunny rooftop patio at Facebook’s satellite office in Seattle, research scientist Matt Uyttendaele showed off just how easy it has become to masquerade as a professional photographer.
Arm outstretched, holding a smartphone-sized 360-degree camera he bought on Amazon, Uyttendaele smiled in the general direction of the camera’s tiny lens.
“It really doesn’t matter where I point because it’s going to capture everything,” he said. “Cheese!”
With the click of a button, the camera did capture everything, from the blue waters of Lake Union to the tippy top of Seattle’s iconic Space Needle. “That’s it. Just one shot,” Uyttendaele added. “If I posted this photo to Facebook, you’d be able to pan around with your phone so that you can experience this spot that we’re standing in.”
The photo turned out great. It also relied on a camera that retails for $350.
:format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/7023625/Still%20Photo%202.JPG)
When Facebook paid $2 billion for Oculus back in 2014, it did so under the assumption that virtual reality would be the next big platform — the mobile phone after the mobile phone. That’s probably not going to happen if capturing the perfect 360-degree photo or video requires hundreds of dollars in camera equipment.
That’s what Uyttendaele is trying to solve, along with the handful of other longtime software engineers that make up Facebook’s new computational photography team. That group was founded late last fall by three veterans with a combined 60+ years building photography tools nearby at Microsoft. Its task at Facebook: Make photo and video features that will make virtual reality stick with the masses.
Talk to anyone in the virtual reality world and they’ll tell you the industry’s greatest obstacle is finding an abundance of great content. And user-generated content — the “shot on my iPhone” kind of photos and videos that keep apps like Facebook, Instagram and Snapchat alive — isn’t really possible for VR right now.
This team wants to change that.
One of Uyttendaele’s first projects after joining Facebook last November was ensuring Facebook could support 360-degree photos, the kind of images that also work inside a VR headset. Today, Facebook announced automated video stabilization for 360-degree videos uploaded to the service. This is designed to iron out the jumps and shakes from 360-video footage to make sure it’s actually watchable.
“360-degree video stabilization makes all the difference,” said Rick Szeliski, the head of Facebook’s computational photography team and another Microsoft veteran. “If you put on a VR headset and the video’s wobbling around, you get motion-sick in just a few seconds.”
Capturing that kind of video is not yet mainstream. (Remember the $350 camera?) But eventually, Facebook believes it will be, which is why the team is also working to bring VR creation to the devices you already own.
Szeliski says the computational photography team is already working on 3-D photo software for smartphones. It’s possible the product could work as a standalone app — à la Instagram’s Hyperlapse app — but it’s too soon to tell how something like this might materialize.
“I think we could write an app today that is very similar to iPhone panoramas, that captures a bit more information and creates a left eye-right eye [stereoscopy experience],” Uyttendaele said in regard to bringing 3-D content to Facebook. “I’m not saying that we’re necessarily going to, but it’s possible.”
Getting VR content right, and simplifying the process for users, will be key for Facebook’s master plan. The VR industry is growing, thanks in large part to Facebook’s bet on Oculus and ensuing commitment to making it work. Facebook is building all of the tools needed for VR, from Oculus Rift headsets to $30,000 360-degree cameras. So it makes sense that consumer creation tools would be on its radar, too.
Szeliski and his team don’t only focus on VR, though. The trio of Szeliski, Uyttendaele and Michael Cohen all joined Facebook with extensive backgrounds building photo software, things like Microsoft’s hyperlapse product and its 3-D photo technology Photosynth. Szeliski and Cohen are affiliate professors at nearby University of Washington in the areas of computer vision and graphics; Cohen also used to teach classes at Cornell and Princeton.
It’s no surprise, then, that the team has already grown from the initial three members up to nine in less than a year. And more are on the way. “Our management tells us that they want to invest heavily in this area,” Cohen said with a chuckle.
The team has a number of other consumer products already in the works. It helped build out Instagram’s Boomerang feature earlier this year; the team is working on stabilizing technology around live video; it also helps out on Masquerade, a startup Facebook acquired in March that creates the kind of face-distorting features that Snapchat made popular.
“We’re just trying to dream up new experiences,” Cohen said. “Once you know where the face is, what it’s doing, what its geometry is, you can now begin to extract expressions. ... It just opens up a floodgate of ideas.”
Facial recognition is also relevant for — you guessed it — virtual reality. Facebook demoed what it called social VR at its F8 developer conference earlier this year, essentially human interactions in a virtual world. At the time, Facebook discussed the idea of users scanning their faces in order to create their avatars. That’s Masquerade’s specialty.
Nothing guarantees these ideas will work, of course.
360-degree photos and videos are new and still rare. They’re novel, but will probably always require more thought and effort to compose than a simple snapshot, and aren’t particularly gripping in the context and noise of the Facebook News Feed. Boomerang, meanwhile, lives inside what is still primarily a photo-sharing app (though the Instagram team is also trying to aggressively push video creation to its users).
But Facebook and Mark Zuckerberg — who often speaks of a time when most of Facebook’s content will be video — are always thinking years into the future. The computational photography team has that same luxury.
“We’re not just gilding the lily, we’re not just making small improvements,” Szeliski said. “We work on the big, high-impact things.”
This article originally appeared on Recode.net.
