:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/56188869/612440016.0.jpg)
On this episode of Too Embarrassed to Ask, Recode’s Kara Swisher and The Verge’s Lauren Goode are joined by Clay Bavor, the VP of virtual reality at Google. The three discussed the current and future uses of VR and the convergence of VR with augmented reality, a result sometimes called mixed reality.
You can read some of the highlights from their discussion here, or listen to it in the audio player above. Below, we’ve posted a lightly edited complete transcript of their conversation.
If you like this, be sure to subscribe to Too Embarrassed to Ask on Apple Podcasts, Spotify, Pocket Casts, Overcast or wherever you listen to podcasts.
Kara Swisher: Hi, I’m Kara Swisher, Executive Editor of Recode.
Lauren Goode: And I’m Lauren Goode, Senior Tech Editor at The Verge.
KS: And you’re listening to Too Embarrassed To Ask, coming to you from the Vox Media Podcast Network at Google right now, actually. This is a show where we answer all of your embarrassing questions about consumer tech.
LG: It could be anything, like, “What is blockchain?” Kara, what is blockchain?
KS: I have no idea.
LG: We just did a podcast on it.
KS: I still don’t know after we did an entire show about something. Something.
LG: I know. If you’re curious, if you’re a fin-tech nerd, listen to last week’s episode.
KS: Please don’t.
LG: You can ask something like, “Do I need a fancy new router or a Wi-Fi extender if I’m going to get decent Wi-Fi in my house?"
KS: Yes.
LG: We’ve talked about that a lot on the show, and we’re going to talk about it in an upcoming week as well. Will Amazon’s new grocery store be called Prime Foods or Whole Echo?
KS: Oh my God.
LG: I came up with that one myself.
KS: You know what I just realized? Everything I buy is from Jeff Bezos now. I literally ... Like I buy my food at Whole Foods and I buy the other stuff at Amazon. All my money are belong to Jeff Bezos.
LG: Congratulations, Jeff.
KS: No problem.
LG: You’ve put together quite a profile, a nice little updated profile of Kara Swisher. My final question is, “How much has Kara Swisher missed me over the past few weeks?”
KS: Not even slightly.
LG: What?
KS: Oh, I’m sorry, yeah, totally a lot.
LG: We’re hot miced right now.
KS: Right, I know that.
LG: This is live.
KS: Not at all. I have missed you not even slightly. My children have been here all summer and I’ve been enjoying my time with them.
LG: Okay, great, maybe I’ll rethink going to your event tonight.
KS: When you leave, I’ll get back to your juvenile antics. All right?
LG: Okay, all right.
KS: Sounds good. We’re doing a show here this week from Google, actually, but for future shows, send us your questions because we do read them all. We got a ton this week on our topic. You can find us on Twitter, or tweet them to us at Recode or to myself or to Lauren with the #TooEmbarrassed.
LG: And we have an email address, tooembarrassed@recode.net, you can also reach us there.
KS: Yes, exactly. Now I said, we are actually at one of Google’s many sprawling buildings in Mountain View, California right now. We’re broadcasting from there.
LG: We are.
KS: Yes.
LG: And we’re talking about virtual reality today.
KS: We are, we are talking about it.
LG: Which we’ve talked about on this show before. Almost exactly a year ago, we went to Stanford University, their human computer interaction lab and we talked with Jeremy Bailenson.
KS: Right.
LG: One of my former professors, about virtual reality. We did a really fun Facebook live.
KS: I remember.
LG: That was great.
KS: Yeah.
LG: But, we’re going to talk to someone else today about it who is quite expert on this.
KS: Someone else. It’s a big topic in Silicon Valley. One of the companies that’s involved in it is Google, and it’s an area that we’re all very interested in. There’s been a lot of investment in the concept, not just from Google, but from ... Apple’s in it, Facebook is in it, they bought Oculus. All kinds of people are making investments in all kinds of things. So we’re very excited to have Clay Bavor here with us. He’s Google’s vice president of virtual reality. Clay, welcome to Too Embarrassed to Ask.
Clay Bavor: Thank you for having me.
LG: That was a nice, long, rambly intro.
KS: I know.
LG: For those people who like to categorize podcasts and say, “Well there’s ones of like, you know, a bunch of guys talking, and then long rambly ones and then ones with whispered hush tones.” We just filled the long rambly one.
KS: Okay. Exactly. Anyway ...
LG: So Clay ...
KS: We’re here to talk about VR with Clay, so you know what? Let’s have Clay talk about what’s going on in VR. But first, talk a little bit about how you got here — and how you got to Google, which has been dabbling in the area for a long time.
Yeah, it was a somewhat circuitous path. As a kid, I was lucky enough to have a computer at home. It was a LC2, had a color monitor. I had Clarisworks, and started making kinda basic computer art when I was in elementary school. I was just kind of obsessed with this idea that you could create pictures, and images, on the computer. And a little bit later, I saw the first 3-D renders. This was pre “Toy Story”; really the very first 3-D computer graphics.
KS: Don’t tell me you had AutoCAD, did you?
No, I had a more basic, less expensive, less fancy tool, like a Ray Dream Designer.
KS: Okay.
And I’d ask my parents to combine my birthday and Christmas and next birthday present to get a 16 megabyte memory SIM for my computer, so I could have higher resolution textures or whatever.
LG: Quite a skilled negotiator.
KS: What were you trying to make? What were you painting? Apples?
Not apples. I was obsessed with this idea of trying to recreate real objects in these programs, and have them look real. And so, for example, I spent a week trying to recreate my dad’s Walkman in 3-D.
KS: Wow.
I’d do renderings of them and show them to my parents, and try to fool them into thinking they were seeing photographs. And just this idea that you could use a computer, and if you understood light and shadow and form well enough, you could create something that looked real. I don’t know why, but I just thought it was fascinating. And, it kinda went from there. I got into photography, panoramic photography, and ended up doing these, small, silly, freelance projects with 3-D animation and rendering as a teenager. And always had it in the back of my mind, and then, I was at Google for many years ...
LG: Doing what? What were you?
I worked on ads, I help integrate AdMob, the mobile ads company, and I spent three or four years leading the product and design teams for Gmail, and Google Docs, and some of our productivity and work-focused apps. But I had started doing some prototyping work with the very first Oculus Rift Developer kit at Google.
KS: This was at Google or on your demo?
This was at Google.
KS: Okay.
LG: Is this your 20 percent project?
Kinda my 20 percent project, yeah, and at the time, it was as is usual, kind of the 120 percent project or whatever, come in literally nights and weekends. I built this weird thing that I call the teleportation robot, where you put on some goggles and basically wake up in another room. What we were doing was matching your movement in one room to the movement of a robotic arm with two cameras on it, in another room.
KS: Oh, man.
Piping the video back to ...
LG: That’s super cool.
It was cool. It was very strange, but enough people saw that that when the first Google Cardboard prototype showed up — Google Cardboard was the thing where you put your phone into it, it turns into kind of a basic VR headset. It occurred to people, “Hey, Clay should see this.” And I thought it was neat, and we had eight and a half weeks to Google IO, and we say, “We should do something with this. This is neat,” and that’s kind of where this all started. So that’s a brief history.
LG: So it’s just your interest? Had Google made the investment in Magic Leap and other things by this time? And Facebook has bought Oculus.
That, it was 2014, and a lot happened that year.
LG: Oh, yeah.
KS: So, yeah.
So I’m not sure the exact time.
LG: Facebook bought Oculus that year.
KS: Yeah.
Cardboard came out a few months after the Oculus acquisition, and I’m not sure when Magic Leap happened. It was all around the same time.
LG: Okay.
KS: Right.
LG: Talk a little bit about Google’s approach to VR. I mean, it sounds like since its inception, it’s something that’s been kind of hacked together, in a sense. I mean, literally your first headset was cardboard, you’ve since done Daydream, which is very mobile focused, and so that’s a very different experience from the heavy duty, tethered headsets we’ve seen from some of your competitors. Talk about why Google is taking that approach.
Yeah, so I think first of all, it’s not that hard to see where this all goes. Imagine you have some glasses, and you put them on and you feel like you’re completely transported somewhere else, courtside at a Warriors game, or at Machu Picchu, or back in maybe a moment of your life that you’ve recorded.
KS: I think that was an episode of Black Mirror, but go ahead. And a bad one, bad episode.
But the reality is all the technology ... We’re at the very beginning of the journey to that fully realized version of VR, and we thought it was important to start simple, where things were low cost — literally, in our case, made out of cardboard. Making use of kind of the tools and technologies already available in your life, and that gave people a way to get a glimpse of the possibility of this thing in a way that was very approachable, affordable and simple. And this insight that, “Oh your smartphone,” with the high-resolution screen and some sensors, has a lot of what you need for a basic VR experience, you can unlock that kind of hidden power of a basic VR viewer with Cardboard.
KS: So is it to get people used to it? It’s kind of ... I’m aging myself, but using Pong to show what video games could do. Remember Pong? You don’t.
I do remember Pong. It was amazing.
KS: Well, it really wasn’t. When you go to play it now, it’s not amazing in any way.
Looking back on it, it’s less amazing.
KS: Right, but it was the first time you could interact with a television and some computers.
I think there’s an element of that, right.
KS: It feels very Pong-y.
It’s kind of a small, approachable, simple proof point of it, and I think for a lot of people like, “Whoa, I’m feeling like I’m there, that’s cool.” And it was a glimpse of something to come.
KS: Right.
LG: Do you feel like you’ve sacrificed quality at all, in terms of the experience, because you’ve ... It’s so mobile focused, mobile centric, right now.
Well, obviously when you’re unconstrained on power and compute and cost and so on, you can do more, right? And so, we’re certainly making different trade-offs than, for example, the PC systems make, and those are deliberate. With Cardboard, for example, one of the interesting things not many people know is ... People asked, “Hey, why isn’t there a head strap on it?” Right? Everything else has a head strap. Well, that was a very deliberate decision because, when you hold it with your hands, you rotate not with your head but with your torso. Your torso moves much more slowly than your head. Makes it far more comfortable to use this thing, when things like ... I hear a lot of people talking about latency in VR.
LG: Right.
It makes it more comfortable, so we tried to make the trade-offs to make it more accessible, more approachable, lower cost, but do that in a way that preserved the quality of the experience, the comfort and so on, and I think we did that with Cardboard. I think we’ve done an even better job of that with Daydream, which is the mobile VR thing we launched last year.
KS: Right. So talk about where it goes, because here you are starting off very simple. I see the point of why you would do it. Others are not. Others are starting rather complex, and one of the complaints, except for really geeky men it seems, they think that it’s okay to wear these massive thing on your head with things popping out the back, in a room, in a white room, with edges. It’s just completely unusable for a consumer. So they’re approaching it from that side, and even in the new Oculus or the various Vive or Vive.
LG: Did you just say Vive?
KS: Yeah. They’re very onerous as a system, to be able to enjoy in any way, unless you’re in one of the great labs where they have them.
So again, I think they’re making different trade-offs in the space, and there’s no doubt that VR today, there’s a lot of friction in it, it costs a lot, and because it’s early days in it that there’s not that much to do in it yet.
KS: Right.
And I think, that’s just being honest about where we are. Where this goes I think ... I get very excited about it, where, if I told you, “Here are these glasses you can put on that can give you access to any experience you could have with sight and sound that happens anywhere on the earth.” You would say, “That sounds great. I would love that.”
KS: Right.
Especially if you could go there with other people, right, and you know, I hope to see some portion of the 1,000 most beautiful places on earth. I’m not going to get to them all, and so how great would it be to basically add to the set of experiences that a person can have. So this idea, that by capturing reality experiences, whether it’s the best seats at a basketball game or a concert, or something else that you care about, and enabling instead of one person to have access to that seat, 10,000, or a million people. I think that’s super powerful and compelling, and something that really gets me excited.
The other is, and I think this is kinda fully realized version of VR, and again, it’s gonna be a while to get there, but one of the other areas is capturing moments in your life, right? If you think about, what do we use our phones for, cameras for, very few of the photos we take are kind of artistically beautiful and well composed and so on. I’m speaking for myself.
KS: Yeah, you are, because we take beautiful photos.
It’s possible. It’s about capturing a moment, capturing a memory, and we have line of sight to kind of the first version of VR cameras, that let you record an experience, and I’ve been using these in my own life. I have two young kids, and you know, of course you wanna be present, but there are these moments. You go to the beach, right, or even just moments with young kids where they’re at a certain state, and five seconds of breakfast, 30 seconds of breakfast. The ability to step back into those moments and almost re-experience them, it’s very powerful. And that’s another direction I think this goes, that I think is super exciting. Then there are the obvious ones, gaming, immersive gaming, and so on.
LG: Sure. Entertainment.
I think that right now, today, and I think will remain for a while, a smaller more niche use for it.
LG: The gaming?
A smaller set of the population is gonna be interested in fully immersive VR gaming.
KS: Oh you could go ... Obviously, besides ... Let’s just skip over porn right now, but the gaming or experiential kinda things, which you’re talking about. One of the things that I’ve noticed when I’ve tried a lot of these things — and I’ve tried everything from the touching the whale one to the gaming ones to watching a full movie where you feel something behind you. I just did one with Jon Favreau, who’s working on a bunch around gnomes, or there’s goblins.
LG: Goblins and gnomes.
KS: Whatever, something, little tiny creatures.
Haven’t seen that one yet.
KS: It’s cool.
It sounds great.
KS: I don’t love gnomes but that’s fine, so they’re all different. That one’s more for kids so it’s not scary, like a lot of them are scary, but one of the things you’re talking about is very different. It’s creating complete reality in your present, like complete ...
LG: Yeah, like creating these sort experience for ... In perpetuity ...
KS: Right.
LG: For people that you know now, which is interesting because one of the companies ... There’s a company in LA that I did a video piece for Recode a few months and they have a Tango app, actually. We’re going to get to Tango in a bit, but one of the thing ... And they’re mostly doing entertainment and marketing things, but one of the things they mentioned as an aside was that there were some people in LA who were coming in to get holograms of their children made because they wanted them sort of captured in this moment in time. So that was super interesting.
KS: That’s creepy.
LG: Wrote on my list of things to look into.
KS: What I’m thinking about, when you think about these things, is you don’t know where it’s going to go. And I’m just thinking of a movie that I reference a lot, which was the Tom Cruise movie, “Minority Report.” And they go to that hall where people experience like, “I won the Oscar,” or all kinds of different things, whether it was porn, or whether it was skiing, or whatever, or whether it was killing your boss, like that kinda thing. I don’t think we can know where it goes. We can guess in a lot of ways, where these things go. How do you guys think about that? Because I know you guys have hired an anthropologist. You’re kind of looking at ... Your news lab is doing some of that, like why people want these things. What do you imagine is gonna break through first of all these various things?
Yeah, I guess to start my hope is, and my belief is, that it’s gonna be tremendously additive to the human experience. What you can do, what you can see, what you can experience, from ... And I talked about things, like the best seat, or whatever you care most about, remembering things important to you. What we’re seeing on Daydream, for example, is that people love going places.
LG: Experiential.
People love seeing stuff, experiencing stuff, that’s hard to get access to, and that could be the other side of the world. One of my favorite VR pieces, by Felix and Paul, they did basically a guided tours of the White House, a place that most people won’t go, given by Barack and Michelle Obama, and you sit five feet from Obama, and it’s a very compelling experience.
LG: I can think of some people who might want to just slip back into that now, if they could. I don’t know, maybe yeah.
KS: Yeah. Yeah.
It’s a great piece of VR content, and I think, “Is it the fully realized version of VR?” No. Not yet, but I think you can see glimmers of it.
KS: Before we get into other competitors that are doing this also, what do you imagine needs to happen, because of the things when you’re talking about that there’s ... I remember 20 years ago the media lab in MIT was talking about haptic, haptic touch, and feeling. I just went to Universal and I was in a movie and they sprayed water at me when I had glasses on, but you’re suppose to feel like you’re in the ocean. It worked. Let me tell you, it worked for the whole crowd. I was like, “Oh, I’m at the ocean.” It didn’t take much. How does that work when you think ... because you’ve gotta have the entire haptic experience. You can’t just be in a visual environment, there’s gotta be sound environment, you’ve got to feel like someone’s behind you. You’ve got to ... How do you imagine that occurring, if just wear glasses? Because you can convince people of it visually but you can’t ...
Every sense adds something, right. And so vision is part of it, and really the ... What’s behind this feeling in virtual reality of being somewhere else — and that’s kind of my cheat sheet definition of VR, which is it can take you anywhere and make you feel like you’re there.
KS: Sure.
Vision is one part of that, and by vision it means, as you move your head and move your body, the world that you see reacts to your movement in exactly the same way the real world would. We can add sound to that; there are technologies to do what’s called specializing audio, where you can make a sound or a voice appear as if it’s coming from somewhere in space. Haptics, as you say, touch and feel, once you see something, you can hear it, and reach out and touch it, it makes it even more realistic.
KS: Right, it pushes back.
That’s right. And so there’s a whole class of experiences possible, without haptics, for example. For example, at the concert watching a musician, you’re seeing and hearing stuff, so it not a hands-on experience.
KS: Oh, you should have beer spilled on you, but go ahead.
Yes, very advanced haptics there. That would be challenging, we haven’t solved that one yet. So I think you’re going to have this large library of kinda sight and sound experiences, and then, a smaller library, a smaller set of possibilities with haptics, where to start it’ll be controllers and simple things pushing back on you. But how you do haptics to something like leaning on a wall in virtual reality, we have no idea.
KS: The wind, water.
We have no idea, right. The technologies to create those things — without extreme expense — don’t exist.
KS: But could you trick people into it? I’m just curious, because sometimes when I’m in those environments, it does trick me. I do feel like someone was near me, but they weren’t.
A lot of VR is about illusions, and right, you in essence trick your senses into thinking that they’re experiencing reality when they’re not. And so there are tricks you can do. So, for example, you could replace the wall that feels like a brick wall with a wood wall that someone rolls in.
KS: Right.
Okay, and while it’s not a great scalable solution, it’s doable, right.
KS: Right.
And I think you’ll find installations like The Void ...
KS: Game.
It’s something that I’m very excited about. These kind of installation VR experiences where they create haptics by modeling parts of the environment but then replacing them in VR with walls that are maybe of a different type, but line up with the physical wall.
LG: Like in the Stanford lab I mentioned earlier. The floor is actually a vibrating floor. So that as you’re experiencing VR, you can feel these sort of shakes of the floor.
That’s right.
LG: It can be pretty alarming if you’re not ... It feels very real.
It does feel very real. I think for things like that you kind of need to go to place where it’s installed. I think it’s going to be a long time before we have a vibrating floor in our home.
LG: I don’t know. Don’t put it past Kara Swisher.
KS: I will.
LG: I wanna ask you about AR, because in addition to everything you’ve been doing with VR, you’ve also been working on something called Tango, which for those of you who don’t know, has been a thing at Google since 2013, and it is sort of a software package, but it’s also kind of a hardware specification, where you guys are working with phone makers to get really sophisticated 3-D augmented reality on a phone. Apple, of course, just announced something some called ARKit last month where they’re doing some pretty sophisticated AR stuff, but just with the software on existing phones. Yours has been more of a hardware approach. Talk a little bit about that. What do you think so far of Apple’s ARKit?
Yeah, well, it’s exciting. And first of all, I get a lot of questions about like, “Hey, VR versus AR?”
KS: I think they’re the same thing. They’re similar.
They’re different points on the same spectrum. I think of the spectrum, I call it immersive computing, I don’t really care about the label, but it’s this idea that you have computing and digital imagery that feels like it’s there. Virtual reality, everything is computer generated; augmented reality, you have bits and pieces of digital information overlaid on your environment.
KS: Multiple reality. MR is what I’m calling it.
LG: There are other MRs too. They’re mixed and merged reality. Just adding to the confusion.
There are many. And so, I see them as closely related and it’s the reason we’ve been doing work at kind of all places in the spectrum.
LG: Yeah.
From VR first, to Cardboard, and then Daydream, and AR with Tango. Yeah. Tango started in 2013 and we had the first kind of mobile AR developer kit, out in 2014, and the insight was, “Huh, you can use mobile sensors.” Cameras on mobile phones, the sensors that give your phone a sense or orientation, you can use cheap consumer-grade sensors to do pretty sophisticated sensing things, like tracking where the phone is in space, which lets you do thing like place an object convincingly on the ground and feel like it’s really there, give you direction through a building, things like that.
And so, I think what’s exciting about ARKit is not so much the ARKit software itself. Those techniques have been around for a while, and they’ve been running on Tango phones for a couple years. It’s the scale of the Apple ecosystem, and the developer pull and interest that creates. We’ve been thinking about investing in the space, thinking about investing in the space for years now and having more people thinking about building for immersive computing; whether it’s AR or it’s VR. I think it’s a great thing and something I’m excited about, so I think it’s real interesting.
LG: But right now, for example, I used the Lenovo Phab Two Pro smartphone but it had to have, I think, four different types of cameras or sensor sets in it, including an IR sensor, a depth sensor, a camera sensor, all of that. I mean, are you thinking about doing some type of solution that wouldn’t require all those sensors to be in a smartphone?
I think scale’s really important. Developers building where a lot of people can use their software, right. And so, I think, overnight not every phone is going to have some of these more advanced sensors in it. I think there are paths to having far more devices than just those with the dedicated sensors, have it running things like Tango, and so that something we’re excited about.
KS: When you think about AR, what do you think the biggest applications are? One of the ones you had — which caused a lot of sensation and sort of went somewhere — was Google Glass, which is now sort of back, apparently. A lot of people were fascinated by it and then, quickly, discouraged by it. I always thought it was super unattractive. It didn’t work really well. It was super glitchy. I had a line I kept saying, that it renders supermodels unattractive. You know, just the whole thing wasn’t a great experience. Is it back? Are you working on it? Are you using it? Or is it just an idea that doesn’t, an idea ... You said glasses several times.
Yeah, Glass ... I think you’re referring to Glass Enterprise edition. Right, there was some news earlier this week about that, that’s actually run from the Google X Team, so part of the Alphabet family but not part of my team. And I think what they’re doing there is smart in the recognition of where Glass found ...
LG: Yeah, people in the field.
... where Glass found it fit. People found it really useful. I think Glass ... The idea and the kinda fully realized vision of immersive computing glasses in AR. I obviously agree with that. I think it was a bit early for its time, I think. And it was not yet broadly useful, but for someone, like physicians where they need to see patient notes and have their hands free, it’s immensely useful, right? And so I think that team is clever to kind of focus on that. I think of it not as AR, more as like a heads-up display that you can use hands free.
KS: So what is AR?
LG: You think it’s more like HoloLens in that way? And it is competing with Microsoft in trying to attract enterprise clients?
I think the thing that distinguishes augmented reality from, for example, a companion screen that follows you around is, for me, awareness of the environment, and the ability to put computing into the environment. So Kara, you asked about, “Hey, what do you think the uses for AR will be?” One I am so excited about is very simple, and that is navigation.
LG: Right.
And so I spent my day in this building. I still can’t find all the conference rooms, right, and so imagine, that your phone ... And you could just look through your phone, “Oh, your next meeting is over there.” Or you get out of a taxi, right, or Uber, and it’s like, “Where am I?” You’re disoriented. It would be nice if your phone was just, “You’re going over there.”
KS: Right, without having to call up the map.
Overlay footsteps on the ground leading the way for you.
KS: Yep. Microsoft made a video years ago. Did you ever see it? Where they do that? Where they were imagining ... I’ll send it to you.
I didn’t but it’s a good idea.
KS: It’s fascinating.
It’s a good idea.
KS: Someone did a joke version of it, which was funnier, but it was an overlay of footsteps like, “This is the way you go.”
It’s a good idea and that will be a thing and it’ll be helpful, instead of getting out of the cab and trying to make sense of our blue dot on the overhead map, and waving our phone to calibrate the compass and so on. It would say, “You’re going over there.”
KS: We do shift our lives for you people. Like we do all kinds of weird and unfortunate things.
What do you mean, “We shift our lives?”
KS: Meaning, I was thinking about that today. I was using the map, and I’m like, “What the hell way does this go? Why is not just orienting to me?” And I was thinking, “The freaking Google people have me doing what they want me to do versus ...” I had a little moment.
So I think AR could ...
LG: You’re in the right place to dump that.
KS: I knew that. I’m complaining. I’d have to call the Maps people.
I think AR could help with that.
KS: Yeah, exactly. So just before we get to readers’ questions, one of the things that I think is important is business. Like Google has enormousness amounts of money and you’ve got your little search business that seems to be still doing rather well and paying for all of this. And besides your hoverboards and time machines, this is probably an area of great promise from a financial point of view. How much pressure is on you to make money at it? Or you can see there’s money to be made here. This is something you can actually see a path towards business and all kinds of content business hanging off of it. When do you imagine that happens? Like there’s obviously devices, there’s content, there’s subscriptions, all kinds of things that could happen. How do you look at this as a business?
To start, we’re doing what Google generally does, which is start with figuring what is the product and user experience, and building something that’s useful to user. And to trying to figure out how should this work, how can we do a useful job for our users? And kind of assume that if we can do something valuable, delightful, fun, then the business models and monetization opportunities follow.
And I think there’s some near-term things that we have line of sight to. For example, we see huge interest in, again, VR video and YouTube VR. Because it’s fun to go places. It’s fun to go to Coachella if you can’t make it to Coachella, or to travel to the other side of the world with Street View. And you can imagine VR ads, where you can go inside the showroom for a new car or whatever. And so I think that’s a very natural extension.
What I’ll say is, we’re very much in the figuring it out and getting it right phase. The product, the technology, the user experiences. And while I think it’s useful to keep on an eye on the business models and how those will play out, it’s not our focus right now. It’s the technology and, most importantly, the use cases, the jobs we’re doing for users, taking them to other side of the world to see something they care about and things like that.
KS: And do you imagine that you need help from Hollywood? Not to say that Google people are not creative, but Google people are not creative in that way. You know what I mean? How do you partner with people who are actually ... Because to me one of the problems with Hollywood is that there’s a lot of people who can make movies or TV shows, but there ... This is a whole other class of content creatives. It’s totally a different thinking.
I absolutely agree.
KS: That’s why I wanted to interview Jon Favreau because he’s sort of thinking that way, and even he has a hard time with it, and he’s obviously ...
LG: Right. Like distribution is changing so much. There are some producers that still don’t necessarily want to work with just the basic streaming platforms because of the way it shifted the economics. So when you talk about something like, “Now let’s make content for VR,” that’s a whole different area of distribution.
KS: And creativity.
LG: People’s heads.
KS: Right.
So Sir John Hegarty of, is it BBH? One of the big U.K.-based advertising agencies, has this great quote, he said, “Technology creates possibility, creativity creates value.” And I like to think a lot of the job we’re doing is solving some of the hard technical problems. From how do you capture reality in a way that you can kind of represent it in another place in virtual reality? Or the glasses, the hardware, that actually makes stuff possible, but absolutely though, the artistry, the content, the storytelling, the narratives, the IP, the characters that you connect with. It’s so important. And the headset, or an operating system, or anything without the experiences, the content, the art in it, is not of value.
KS: So we’re just gonna do the Google thing? They’re going to pay for the way YouTube sort of tried to seed ... Because you don’t just get “Star Wars.” Although George Lucas did do the special effects stuff and then used it for a story that made billions and billions and billions of dollars. Who does that? Is Google gonna have to pay for creators to exist?
Well, I think it’s a combination.
KS: Or Facebook or whoever.
I think it’s a combination of things. So one thing I’m really proud of that we’re doing is we have ... We built this system called Jump as in, enable you to jump to Paris and then the other side of the earth and Machu Picchu, and it’s a camera system for capturing VR video, very high-quality VR video. And we partnered with a couple companies to make these VR cameras, and we have a program called Jumpstart, which we’re loaning cameras and giving processing time to students, artist, and so on, to go and use these tools and see what they do.
So it’s kind of enabling creative people with the tools, and helping them with the technology, with the process, in some cases, with money, to go and make great things, and see what happens. With Tilt Brush, we had an artist in residence program where we reached out to great artists and said, “Hey, here is this tool, we think it’s neat. What will you do with it?” And people came back with some amazing things. And then, I think, of course as with any kind of medium, we’ll be looking to partner with Hollywood and developers of games to bring franchises and IP and so on into VR, into AR, so the people can connect with characters and stories and stuff that they care about.
KS: Yeah, I don’t know what it’s gonna be in Hollywood. That’s the thing, you know what I mean? We go there immediately but it’s a whole different mentality, I don’t think it’s ... They’re trying. They’re trying, but it’s an interesting question of who ... Because that’s what’s gonna matter where VR is concerned.
I think Hollywood will matter a lot. I think a direct translation of film to VR won’t work but I think a lot of the instincts for story, character, special effects, crafting alternate realities, I think that’s really useful.
KS: That’s interesting.
I think in addition, you’re gonna have people who’ve been building games. Games are in essence VR worlds, we just didn’t have the goggles to inhabit them, right? And so I think both are gonna matter a lot, and I wouldn’t short sell Hollywood at all in this area.
KS: All right, all right. In a minute we’re gonna take some questions about virtual reality from our readers and listeners and Clay is going to answer them? Right, Clay?
That’s great, I’m excited about this.
KS: But first we’re gonna take a quick break for a word from our sponsor. Lauren, the word, please.
LG: Ka-ching.
KS: Thank you.
LG: How do think that’ll sound in 3-D sound? It will be in our headsets.
KS: Money will fall on your head in a haptic fashion.
LG: Yes.
KS: Some will throw money in people’s faces.
[ad]
KS: We’re back with Clay Bavor talking about virtual reality and now we’re gonna take some questions for our readers and listeners. Lauren, do you want to read the first question?
LG: You know I do. The first question is from Kevin Swint, @Kswint on Twitter. “Is the primary barrier to mass VR adoption simply the form factor of the headsets?”
Oh, hey, Kevin, thanks for the question. I think it’s a bit more complicated than that. Certainly the headsets are part of it and I guess it’s important to remember that today’s headsets are largely made out of components that were really meant for smartphones. So VR headsets basically contain a smartphone display, sensors from smartphones, and so it’s kinda like trying to make airplanes out of bicycle and car parts, right? You can do it, it’s possible, that’s how the Wright brothers got started, but it’s not exactly where things converge.
So the headsets are part of it, but also I think we’re still learning about the content and the medium and the experiences and what works and the tools needed to create those and so ... I think more than just lower-friction, higher-resolution headsets need to come together. But the cool thing is we see progress in all these fronts. We’re figuring out what works and improving the hardware, so I think we’re heading in the right direction.
LG: Do you think that even ... I’m gonna ask a quick follow-up question. Even if you got this engineered perfectly, there is incredibly high fidelity and a great experience. Do you think people will get used to sitting, let’s say next to each other, on a couch at home both wearing headsets?
I think there are two parts there. One is getting comfortable with any technology. I think if you went back 20 years and you told people that everyone is gonna walk around the street looking at a small glass rectangle in their hand, they might be a bit confused. And so I think getting used to it is part of it. I think even more important is there’s not yet great software for enabling you to go places in VR with other people in the same room. Going to concerts is a lot more fun with friends, right? And so we aren’t seeing those kind of shared experiences yet and I think that’s one of the big breakthroughs that we’re gonna have to make.
LG: Great, next question. Kara, would you like to ask?
KS: This one’s from @Rohanbhade. “Apple’s betting on AR, Google on VR, thoughts? Also, would Daydream work with iPhone someday? P.S. Kara, don’t try my last name.” Oh, I just did and I failed badly.
Yeah, so great question. To start, I don’t see VR and AR as these two separate things — we talked about this a bit — but rather kinda points on the spectrum of immersive computing and I think of what we’ve been doing. Google is working along the spectrum in VR with Cardboard and Daydream and more in AR with things like project Tango, which puts kind of augmented reality capabilities into smartphones. Anyway, we’ve had those out for two or three years now. And so, I think we’re betting on immersive computing, that that is going to be useful and unlock some really, really powerful tools and experiences for people in a big way and, again, where and how and when and exactly form-factor first. It’s hard to say exactly but I think we see that being very important in the future of computing, entertainment and other things, so that’s how I think about that.
Daydream on the iPhone, I never say never, but it’s important to realize that a lot of the work that we did to make Daydream run on Android phones is a kinda VR-enabled Android phone, was deep in the operating system, the core software running on the phone. And, of course, we’re not able to work on the iPhone operating system on iOS. And so barring some pretty major changes there or some other unforeseen thing, I think it’s pretty unlikely that Daydream is gonna be on the iPhone.
KS: Okay, next question is from @Zduboss, hashtag #Tooembarrassed. “What’s happening with the standalone Daydream headsets? Will they be comparable to the Vive or Rift?” Are you going to tether Daydream headsets?
Yeah. So at Google IO — which is our developer conference — back in May, we announced what we call StandAlone VR headsets for Daydream. And that just means that rather than having to hook your VR headset up to a PC or put a smartphone in it, everything that you need for VR is there. So the displays but also kind of the computing, the graphics processor and so on, all on these goggles. And so those will be rolling out ... The first of those will be on sale later this year. And no specific updates there right now but later this year.
They won’t be compatible with ... Sorry, they won’t be comparable to the Vive and Rift, and that you won’t have to plug them into things. And so that’s very different. And of course, they’re not relying on a big beefy PC for power and compute and so on. And so the experiences that are gonna possible, and then we’re gonna be a bit different. But we see some very, very compelling things and we’re excited about just how much easier these things are gonna be by being self-contained. You pick it up, you put it on, you’re in VR. We’re looking forward to that.
KS: So there’s lots of questions about nausea and dizziness. This one is from Maria Petrova. “What’s the part of the brain that causes dizziness from VR, motor cortex? Do universities and hospitals actually use it for surgery?” Next one is from Todd Heberlein: “Does Google recommend frequent breaks from VR? If so, how frequently?” Dr. Bavor, you need to tell us. “Any evidence that hurts VR’s appeal to hardcore gamers?” And then Michael Welch asks, “Ask them what the worst, funniest incident of motion-sickness from VR they have ever seen or heard.” So Clay, have you thrown up? I think that’s really the question there.
Well, it’s quite a set of questions, very exciting set of questions. The first, so what can cause discomfort in VR, motion sickness or feeling a bit off? VR done well shouldn’t do that, and by and large doesn’t do that. But when it’s off, what’s happening is what you see, your vision, disagrees with what your head feels, and, in particular, what your inner ear senses. And so there’s this weird thing in your inner ear, which is basically a kind of biological orientation sensor. It’s what lets us keep our heads level with the horizon and gives us our sense of balance and so on. And our brain is used to as we move our head, you experience the orientation change of your head through your inner ear. And your vision, what you see through your eyes, agrees with that. When those two don’t agree, so for example, if in VR you turned the scene without the person turning their head, those two senses would disagree and that’s what causes the discomfort.
Again, done right, VR doesn’t do that. That’s one of the things we’ve been working hard on. Things like what we call tracking, where the head ... We track the position of the head and make sure that the imagery lines up perfectly with how you’re moving.
Let’s see, the other ones. What have we seen ... The worst, funniest incident. Well I’d say ...
KS: For the frequent breaks, actually, do you recommend people take breaks from VR?
Oh, I think mostly it’s listen to your body, right? Listen how you feel and if you feel off, okay take a break, just as you take a break if you’re watching TV for a long time.
KS: So there’s no mechanism in Daydream or anything like Google’s right now that will say, “Okay, it’s been an hour, you should take minutes off.”
Not that I know of.
KS: Okay.
Not that I know of, no. And then, worst funniest incident. I can tell you one really interesting thing we see is in an experience that we put people through if they’re kinda seeing VR for the first time. We call it the diving board and it’s very simple, you put on goggles and, remember, you’re just kind of in a conference room or an office building in Mountain View generally. You put on goggles and you’re at pool side and it’s very clearly not a real pool, you look down at the ground and it kind of is made of these white rectangles and you look to your right and there’s a diving board. You look up and there’s an even taller diving board, you look up some more and there’s an even taller diving board. And I think the tallest one is 50 meters up. And so we push a button and you’re kinda teleported to the top of this 50 meter tall diving board and what happens next is amazing.
They duck down, they lower their center of gravity, and apparently we’ve evolved to ... If we find ourselves on this face of a cliff to lower our center of gravity so we don’t fall off, people will reach for hand railings that aren’t really there. And once they kind of get their bearings, they stand up and then we say, “Great, so could you please go to the end of the diving board and take a step off.” Remember, you’re just in a conference room in Mountain View, everything’s fine. People kind of inch out to the end and they look over and most people cannot step off. Most people can’t do it. Knowing full well that it’s not real, having the imaginary clearly being not real, they can’t do it. I think it just speaks to the power of VR to really make you feel like you’re somewhere else. Those are kind of the funniest, most interesting experiences I’ve seen. Fortunately, you know, we don’t have anything dramatic than not being able to jump off the diving board.
KS: I kinda felt my stomach drop a little bit as you were telling that story, so ... and I’ve had a couple of experiences like that in VR, but we’ll save that for another podcast. This question is from Alex Hardy, who has written in before. Thanks Alex, @canthardywait on Twitter. “What’s the next billion dollar VR use case and what’s the next billion dollar VR company?”
Well, first thing it says, I think we’re still looking for the first billion dollar use case and the first — maybe the second after Oculus — billion dollar VR company. One of the use cases I think is just going to be so powerful and ubiquitous because everyone cares about something. A team, a sports team, an artiste, a place is the idea of taking other places, kind of teleportation with VR to other places around the world. I think that’s very promising as one of these big billion dollar use cases.
Next billion dollar VR company, I have no idea. I have no idea. What I can say is I think there is so much potential in this space where you can put on some goggles and really feel like you’re somewhere else, whether that’s some place real, some place created, something artistically beautiful, something you care about, something you are learning about. So I don’t know what the next one will be, but I’m sure that there are going to be many of them over the next decade.
KS: Next question is from Sandy Ressler, @sressler, who asks, “Any plan to support more capabilities to live in VR environment to do daily work, typing well, editing docs, etc.?” Similar question from Gummi Joh, I hope I pronounced that correctly. “What actual use cases does he see for VR in general other than fun and games in the next five years, like in the professional field.”
On Sandy’s question on typing, editing documents and so on, I think our computers do a good job of that now. What I always look for in VR are things that you can only do in VR, uniquely do in VR, as opposed to things you can also do in VR. And so I think you could do that, but should you and is it better than using a laptop or a tablet? I’m not sure, and so I don’t see that being a big use case in the near term, and that’s partly because our existing tools do that well and secondly because, for example, it needs a lot more screen resolution in VR than we have today.
On the second one, what use cases do I see beyond fun and games like in the professional field, we already see glimmers of this in an application that we’ve actually built called Extraction. It’s a tool for teachers to take classes on virtual field trips to anywhere, that kind of the virtual reality version of it.
And a few months ago we announced an augmented reality version of it where you can bring any object, the statue of David, model of the Colosseum in Rome, into the classroom. It turns out that virtual and augmented reality, immersive computing, is a tool for teaching, is a really powerful tool, and I see that extending into the professional fields in things like training, in simulation. I think VR will be pretty important and valuable in that space.
KS: It’s probably a good segue to the next question from Peg Achterman, who says, “I’m just beginning to teach some VR to college journalism classes, suggestions for good use and stories.” You mentioned an education use case, but here’s ... what’s a good example? I’m curious about this too. I don’t know what a good story would be for someone to explore VR as a storytelling possibility.
Well, I think great journalism helps explain what’s going on, how to think about something. Great journalism can also help you imagine what’s going on in another place, kinda help take you there, describe the setting, describe the scene and so on. And I think it’s in that second one where VR can be especially helpful. There is ... I think there is a big difference between reading about a place or actually having a glimpse of what it’s like to physically be there.
And I think we see news outlets using VR to do exactly that. Put you alongside combat troops, right? For example, oh, okay, wow, that’s what it’s like or to put you in the middle of a parade or festival or protest. And I think that’s where I would focus if I were using VR for journalism, taking people places, helping them fully get what it is to be there.
KS: At the version we have, we did a 360-video at one point of Michelle Obama and it was fantastic. But the one thing I would say is — and our experiences with it — is that it’s very production intensive, it’s not “Let’s just grab our DSLR and run out there and shoot something simple.” It takes a lot and it takes a lot to edit as well.
We’re working on making that easier.
KS: Oh good, great.
With great companies that make some great tools.
LG: Great. Kara do you want to ask the last question?
KS: From Edward Viero: “Here’s my question. Do you believe VR can be the drug of the future due to psychological addiction?” So, Clay, are you a drug dealer?
I think about this like I think about our phones or TV. Our phones are engaging, TV is engaging. I think VR has the potential to be engaging and entertaining and amusing in a very similar way to our phones or TVs, and so do I see it somehow being qualitatively different from those. I’m not sure I do, right. And I think you’ll have people that use VR for really useful things, like people who use VR for entertainment, and I think just as our phones and TVs are engaging, it will be engaging, too. You have people who use VR a lot probably, right? But I see that as no different from we all use our phones a lot.
KS: But you are also promising that it will be more immersive than that. The promise of VR is that it gives you a more realistic or immersive engaging experience, right? So I guess if you’re impacting, I don’t know, human emotions or cognitive development in a greater way.
I agree the sense of immersion and presence and the kind of richness of the sensory experience, that’s absolutely the promise of VR. I think we have some work to do before we fully realize that role.
KS: This has been another great episode of Too Embarrassed to Ask. Clay, thank you for joining us.
Thanks so much for having me.
This article originally appeared on Recode.net.
