Full transcript: VR researcher Jeremy Bailenson on Recode Decode

On this episode of Recode Decode, hosted by Kara Swisher, Jeremy Bailenson, the director of Stanford University’s Virtual Human Interaction Lab, talks about his new book, “Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do.” His work in VR is aimed at providing meaningful experiences that increase empathy.

You can listen to the entire interview here or in the audio player above. Below, we’ve also provided a lightly edited complete transcript of their conversation.

If you like this, be sure to subscribe to Recode Decode on Apple Podcasts, Spotify, Pocket Casts, Overcast or wherever you listen to podcasts.

---

Kara Swisher: Recode Radio presents Recode Decode, coming to you from the Vox Media Podcast Network.

Hi, I’m Kara Swisher, executive editor of Recode. You may know me as someone who believes we are living in the Matrix, the news is so ridiculous that someone has to be writing it, but in my spare time I talk tech, and you’re listening to Recode Decode, a podcast about tech and media’s key players, big ideas and how they’re changing the world we live in. You can find more episodes of Recode Decode on Apple podcasts, Spotify, Google Play Music, or wherever you listen to podcasts, or just visit recode.net/podcasts for more.

Today in the red chair is Jeremy Bailenson, the founder of the Virtual Human Interaction Lab at Stanford University, where we are taping this podcast. He’s also a professor in Stanford’s Communication Department, and is the author of a new book called “Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do.” Jeremy, welcome to Recode Decode.

Jeremy Bailenson: It’s an absolute pleasure to be here. Thank you.

Good. So, Jeremy and I met on a panel. I’ve known of you, of course, for a long time, about Common Sense Media and the impact of virtual reality on kids, and we’ll get to that in a minute. And we just went through a short version of a virtual reality thing that Jeremy’s working on, which we’ll also talk about, which is around empathy, and the actual, maybe helpful uses of VR versus just entertainment.

But let’s get a little background on you and how you got to here, and what you’re doing, and what Stanford’s doing in this area, because it’s a growing area, obviously, within Silicon Valley.

I’ve been working in virtual reality since 1999.

So you’re one of the early ones.

And my PhD was in cognitive psychology, and in 1999 I looked around and I realized that I didn’t really love what I was doing. I was building models of how the mind worked, running experiments on people and writing computer programs to represent the mind, and I decided, “I wanna leave my field.” I was lucky enough to get a postdoc at UC Santa Barbara, where I learned how to build VR from a hardware standpoint.

Why VR? What got you ... Sorry to interject, but.

No. So one of the reasons I got into VR was an amazing novel called “Neuromancer,” science fiction by William Gibson.

Affects a lot of people.

It was, you know ... I realized as I was trying to build artificial intelligence, I wasn’t that good at it, but with VR you can fake intelligence, you can create an illusion that causes people to really feel like they’re in a place, and I was inspired by “Neuromancer” ...

What was inspiring, because I thought you were gonna say “Ready Player One,” which is about to become a movie? That’s another one.

I just re-read “Ready Player One,” and it’s awesome, but to me “Neuromancer” is the true bible.

Because why? Tell me why.

It really pushes in a world where VR is, what does it mean to be a person, what does it mean to be co-located with people. Unfortunately, it’s a very dark vision.

They all are, yeah.

Yeah, if it bleeds, it leads, right?

Yeah.

But in terms of, especially ... You gotta remember, Gibson writes this in the late ’70s, and when he writes it, he’s working with Jaron Lanier, who is the godfather of VR, created the term, and there’s this synergy where the really early demos of VR, he takes these into account as he’s writing it. In my opinion, just really sets up a place to understand what people are ...

So coming from cognitive psychology, you’d be attracted to this, re-making people the way they are in a different place?

In a world where there’s no rules, what do people do? That’s really what got me excited.

Right, and there were the games. There were a lot of these games that were in that genre and in worlds that people would create in games, even those ... All the different games, all the various Dungeons & Dragons games were a version of that.

The first VR I ever did was in 1994. I was interviewing to be a grad student at Berkeley, and on the Embarcadero they had this game called Dactyl Nightmare. It was running 10 frames ... It was running at 10 frames a second, the tracking was off. It was still one of the coolest things I ...

What did you do in it? Dactyl Nightmare.

You stood on this platform, and they tried to do a network, and they were basically combating people and dinosaurs. And to be honest, all the details are a little fuzzy.

Right, because you liked it. You thought, “Oh, cool.”

But it was enough to make me think that ...

What were you wearing on your head? Like a big giant helmet?

The helmet was not big and giant. It was certainly bigger and gianter than what we have now, consumer grade.

It’s still big and giant, yeah.

But nothing compared to the monstrosities that I later put in my lab upstairs.

Right. That’s what I thought. So you started doing that. There weren’t that many people in the field, although it was talked about a lot. There was a lot of attention towards Jaron and others.

There was a handful on engineers that were doing it, and then the reason I got this postdoc at UC Santa Barbara is because we were using it to study perception. So if you try to think about how the human understands vision, how we understand sight and sounds, VR’s a great tool, because you can, say, dissociate what the eyes see from what the body’s doing. You can have a person physically walking but not see the updates or vice versa, and it’s a really nice way to understand the visual system.

So I was lucky enough to get this postdoc at Santa Barbara where there’s, again, just a handful of people looking at psych and VR. I was lucky enough to shift out of perceptual psychology and work with a guy named Jim Blascovich, who taught me the social world, how to ask bigger questions about social interaction and communication and training, and what began was a really fun collaboration where we asked the question, in a world where there’s no rules. You can change your age, you can look at two people at once, you can have your avatar mimic somebody. When there’s no social rules, what happens and how does it change the world?

Right. And so, one of the things, meanwhile, as you’re doing this, the Internet’s starting to explode, really. I mean, there was a downturn, but it pretty much was on the up and to the right, essentially, people using it, with some focus on VR, but a lot less, because VR and other artificial intelligence were sort of hot for a while, and then weren’t, because people more focused on portals, and then social media and Twitter and things like that. What happened during that period, would you say, because it’s made a comeback, let’s just say?

So from my perspective, VR is growing in that period, because when I get there, there’s 10 people, social scientists that are even delving at all into this, and from my perspective I never thought it was gonna be a consumer product this soon. That wasn’t the frame that I had. For me it was, “Wow. We’re actually getting to publish in places that are reputable,” and “Whoa, Stanford’s bringing me out for a job interview. What’s wrong with those crazy guys? Why would they interview somebody doing something so strange in 2003?” So from my perspective, it actually was growing, but I didn’t have the consumer veil on.

Which it has, which it has moved into. So let’s talk about this, just so you define for the people who don’t know these terms, people mix them up a lot of the time, virtual reality, mixed reality. Talk about what the differences are right now. When you say virtual reality, what do you mean versus other things?

Virtual reality is complete transportation. We block out light from the physical world, we block out sound. All of your senses get replaced, and it’s really as if you went somewhere else psychologically, and you don’t see the physical world.

Right, and right now typically it relies on eyes, although there’s haptic stuff, and there’s pushback, and some smells at some point.

In my lab we do always sight and sound. We do a little bit of haptics, and we used to do a lot more, where you basically get force feedback, so you feel touch, and we do a little bit of scent. I’m happy later on to talk about our smell study, or doughnut study, and how it relates to eating.

Oh, sure. Absolutely. So virtual reality is just being transported elsewhere. Augmented reality is ...

Augmented reality, the best way to think about AR is multitasking. So augmented reality, most of what you see and hear comes from the physical world, and we put a digital layer over that. So, if there’s a crowd of people, everybody can have a name tag over their head that only you can see, and you’ll know their names.

Right, so it’s imposing digital things, like Pokémon, I think, or what Ikea’s doing around clothing and things like that. I mean, around furniture, seeing it in your living room. Then mixed reality. Is that different, or is that both of them together?

Mixed reality, my humble opinion, is a term that’s been created to make the world more confusing.

That’s what we do as journalists.

Well, I think it came from a corporation first, but I won’t name that corporation. Mixed reality is, if you think of VR as all digital, then AR is mostly physical light, then mixed reality gets you somewhere in the middle.

Right.

So it’s basically, you can think of it as a continuum. How much of the light are you letting in from the physical world?

Right. Well, let’s stick to virtual reality because that’s where you work in, but when you think about virtual reality, it did go commercial. So you stayed at Stanford, and you were studying what here? What were you hired to do? You did your postdoc studying what?

We are here in the Department of Communication, and when I arrived here, what I tunnel-visioned on for eight years straight was social interaction. What happens when two people network into VR and they see each other’s avatars, and what changes in terms of how they talk, how they feel connected, what are the implications of forming friendships online, the social interaction. Because you have to get tenure here at Stanford, and the way you do that is you just become as good as you possibly can on one area.

And why this? What was your thinking in this area, besides tenure? Jeremy, I suspect you have other motivations.

Well, that’s an interesting ... We’ll talk about the trajectory, but post-tenure, my trajectory changed drastically. So pre-tenure, it was work that I had thought was amazing and rigorous, and I’m very proud of it, but it wasn’t necessarily what I wanted to be doing. So the early work, the social interaction, I’m in the Department of Communications, so studying how people communicate seemed like a good fit. And remember, I came from psychology and came to a new field, communication. So I wanted to try to stay as far down the line as I could.

Now, when I get tenure in 2010, that really frees me up to do work that may not be as publishable in the short term, but it was a little more out there about things that I wanted to do. For example, this is the work we do using virtual reality to teach about climate change, or to teach about empathy, or race relations.

Yeah, we’re gonna get into that, because I just think ... So the idea is using VR as a social tool to improve social justice, really.

The early work was about what happens when you put people together in VR, and I still do that work, and we can talk about it, but ...

Yeah, I do wanna talk about it, but first ...

Post-tenure, we shift to what happens when you put people in places, in places that teach them ...

In situations.

In situations.

Right.

So it didn’t need to have the social ...

Well, role-playing has been around forever, but it’s usually, you’re sitting in front of someone who doesn’t look like what you’re supposed to be or whatever. It’s hard to do, because it requires huge amounts of imagination and shifting.

Whenever we build something in VR, we always go back to the old work and we say, “What was the best way to do it before?” And we don’t try to reinvent the wheel, so we certainly look to the role-playing work.

All right. Let’s talk about where VR is now, and then we’ll talk about what you’re doing now.

So this book you’ve written, this is what you’ve been doing, is looking at that, and then shifting the focus away from just, this is what people do when they’re in VR. But I would like to know, what do people do in VR?

Luckily here at Stanford, we are a revolving door, and everybody comes here to see what we’re doing, but also to show us what they’re doing. From my perspective, there has been a tension, which is that the corporations ... Their job is to make money. They make money when everybody is using VR and they are using it all day long. So that’s why you’re seeing film and media and video games. In my experience — and I have been doing this for 20 years now — VR doesn’t work for these long durations, using it every day, for a couple of reasons.

For me, it’s been watching the companies who think that they can just do everything they have always done in VR come to grips with the fact. And of course no one listens to me when I politely suggest that they have got to learn on their own, there is a reason people aren’t playing video games for 10 hours a day in VR, or why you haven’t seen a feature film that anyone has gone to, because VR is not about, in my opinion, long durations. It’s not about something you use all the time. It’s for these very intense, teachable, aha moments that ...

You just did a demo in my lab, Kara, and you got it. It took you about two to three minutes, you didn’t need to be in there for 20 minutes.

Right, right. Yeah, you could. You could just do that endlessly, but there is a point where you get ... It’s interesting when I try VR now, it is exactly that. I’m like, “Okay. I got that.” I don’t wanna be with the gnomes or I don’t wanna touch the whale anymore, I got touching the whale. Some of them are more appealing than others, like if you’re in Hawaii or on a boat, it’s kind of cool, but eventually that’s enough.

What we’re trying to find as a field of VR is ... You know, when you and I were on the panel together, you made a, it was a funny comment about, “Who would actually wear these goggles?”

Yeah.

And the answer is, if the content is good enough, you will do it.

Of course, yes.

And if it’s not good enough, yes, sometimes it is, and we can talk about the cases. So my ...

I was talking about commercialization. Most people wouldn’t just ... It’s not affordable and hard to use.

I totally agree. I totally agree. But even if you’re going to a shopping mall or museum, the content’s got to be good enough to justify messing up your hair, getting those lines around your face and having your buddies take pictures of you and making fun. So we try to focus in the lab on content that’s worth doing, but not all content is that.

Right. So you’re focusing not on the entertainment aspect of it now?

Look, I’m all for entertainment. And when VR works for entertainment, then I think it’s gonna be great. We’re choosing to focus our lab’s energy on how to use VR to try to solve some harder problems.

On harder problems here. And what is the interest among commercial companies when you’re doing it, because they wanna what? Make movies? I mean, we can talk about why Facebook bought Oculus or why different companies are involved in it, but everybody is. Google is, all the internet companies for sure, and the entertainment companies are certainly dabbling.

Look, the companies come here and visit and all of them have a wing ... You can name, which big one ... They all have a wing, VR for good or VR for social. So they’re all playing in this space. And I actually think that the people who I’ve talked to, they care and they’re doing it for the right reasons.

That being said, that’s not gonna be how their business succeeds. Their businesses are gonna succeed because you read the news in VR and you’re watching your sitcoms in VR. So we spend a lot of time talking to them. We give endless demos to groups from different companies. And I talk to the leaders of these companie, and I get on my soapbox and they politely listen to me about, “You shouldn’t use it all day. It’s not for all types of content.” And we’ll see what happens.

They’d like to see it and hear it all day. Anyway, sitting here talking with Jeremy Bailenson, he’s a Stanford professor who is the founder of the Virtual Human Interaction Lab. Why did you name it that? Virtual Human ... Well, how virtual humans interact.

So no one ever asks that question. I’m glad you did. It was a careful process, because I wanted to ... When I first got to Stanford, I couldn’t be seen as the VR guy, because VR was not a thing, right?

Yeah, yeah. VR guy. Is that an insult? “Oh, it’s the VR guy.”

Like, “What does that weird guy ...”

“Let’s move quickly from RT so he won’t come and sit with us.”

Look, had you met me in 1999, we would not be having this interview. “What do you do? That’s a ridiculous thing.”

I knew what it was.

I know you knew what it was, but I wouldn’t be worth two hours of ...

I was much more interested than ... I was at MIT and they’re doing all that stuff at MIT around it, but go ahead.

Of course. So I couldn’t be the VR guy. So Virtual Human Interaction really points at that social collaborative nature.

I see. So it’s humans but virtual.

That’s right.

Okay, got it. All right. Anyway, here’s the new book out called “Experience on Demand.” Can I ask you that also before we get to the next segment? “Experience on Demand,” what does that mean?

The advice, if there’s one take-home piece that listeners can have, is VR has done well. It’s not a media experience, it’s an actual experience. Our studies in the lab have shown the brain tends to respond how you’d expect it to with a real event. So as you create your content, as you choose whether or not to do content, think, “Would I want to do this in the real world?” The healthy way to think of VR is as an actual experience, not a media experience.

Yeah. It’s interesting when you think about that. I wouldn’t wanna jump out of a plane in the real world. I’d like to do it, but I wouldn’t. So I’d like to do it in a virtual ...

So it’s a great point, and the distinction is could versus would. So I can’t jump out of the plane in the real world, but I would in VR.

Yes, exactly.

I can murder somebody in the real world, but I wouldn’t wanna do it. So it’s about, you should do impossible things in VR. You shouldn’t do things you would do otherwise.

Yeah. I didn’t know you want to murder people, Jeremy. All right. We’re here with Jeremy Bailenson, “Experienced on Demand: What Virtual Reality Is, How It Works, and What It Can Do” is his new book. He’s a Stanford professor. When we get back, we’re gonna talk about where virtual reality is and why so many of the internet companies are dabbling in it, and where we are in the process.

[ad]

We’re here in the red chair with Jeremy Bailenson, the founder of the Virtual Human Interaction Lab at Stanford University. That’s a mouthful, Jeremy. He’s also a professor in the Communication Department and the author of a new book called “Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do.” And we’ve been talking about how he got to virtual reality.

Let’s talk about where it is now. Now, Facebook made an enormous investment by buying Oculus. Magic Leap is another ... There’s just a lot of interest in Silicon Valley over this, although others are not as interested. I was just recently at Apple and they’re much more interested in AR. That’s their focus, the phone being the center of it. Can you just talk about the state of play right now? And also the academic state of play, because that’s different than the commercial one.

Well, with Apple, the first rule of Apple is you do not talk about Apple. So I’ll leave it at that without ...

Oh, dear. You must stop talking about Apple.

Samsung is all in. I’m on their advisory board. I spend a lot of time working with them to help them think that through. Microsoft hired ... So the genius Mark Bolas who was a USC professor, there really Oculus was born because some people go in and sit in Mark Bolas’s lab. He’s now at Microsoft. Of course, Jaron Lanier, he’s at Microsoft. The Chinese companies are all dedicating nine- to 10-figure budgets in this. Sony, of course, is the first person to really deliver to living rooms with a PlayStation VR about 10 million ...

Yeah. Sony’s not a person, but go ahead.

There’s a lot of energy in that space.

Yeah. And why? What is the thinking? You’ve dealt with all of them in the very ... as they enter the picture. What is their interest?

Kara, if I knew the why, I would be wealthy and famous. I think they’re struggling with the, “What do we really want people to do in here?” You brought up Pokémon Go. When you look at the commercials for augmented reality, it’s, “Help me fix a sink,” or it’s, “Let’s understand how to learn this,” and then you build it and it’s, “Let’s play Pokémon Go.” So I think it’s a real challenge.

It’s a lack of imagination, Jeremy.

Wozniak, he talks about when he and Jobs created the personal computer, how they got it wrong in terms of what they thought about use cases. Now, Facebook, Oculus, they thought games was gonna be the home run king. And in my opinion, games are probably better in the current state they are in VR, and you’re seeing at least slow progress there. So why they’re doing it, there’s this kind of sense where everybody feels there’s something here. It’s a transformational experiential thing, but no one’s really figured out why.

And what is the best explanation that you have when you think about it? And we’ll get to the empathy part. I agree with you, I think what you’re doing is much more important. What is the why that’s the best reason for a lot of these companies? Because entertainment companies have been dabbling, not as much as the tech companies obviously, because it’s a heavily tech product, essentially.

Training to me ... What has been the one case that has persisted for the last three decades? And that’s the military using VR to train soldiers. And when you take that lens and say let’s use VR, not just to train soldiers, but let’s train athletes, and let’s train people who work at a big consumer ... So I think training is a low-hanging fruit.

Mm-hmm. In terms of what it’s like, or be in situations, or put them in ...

Last year I co-founded a company called Strivr, and Strivr began using VR to train quarterbacks, to teach quarterbacks how to look around, recognize a pattern. Last year we trained over 100,000 employees of Walmart. And what they were training are things like, look around at our safety violation, is our sharp knife out. Look around during holiday rush with everybody coming at you and practicing coping strategies in these really intense arousing conditions. And this was a use case of the many, many things that Walmart trains its employees. There’s literally a couple-hundred-page document, we chose four or five things out of there that actually we thought would be a good use case in VR.

How expensive was that? Because to create the VR is very ... We just went through on empathy, which we’ll talk about in a little bit, but it’s very expensive to create each of these, correct? Or is the price coming down? Or ...

In VR there’s three ways to do content. One we won’t talk about, because it’s not ripe yet, and that’s called lightfield capture. But putting that aside, you either get to build 3-D models using computer graphics, or you shoot in what’s called 360 video. And 360 video is cheap, easy. The problem is it’s not interactive, so you can’t grab an object. When you’re going to computer graphics, that’s where it gets more pricey and more expensive. And so depending on what your needs are, how interactive you need it to be, it’s gonna get more expensive.

Is that what’s stopping the development? Because you could do these all day long for companies, presumably.

So if you were to ask me six months ago what is blocking this from getting everywhere, six months ago I would’ve said position tracking, which is the way you measure how somebody moves physically so you can update the virtual scene. I think that that’s not been solved, but there’s been enough progress there that I don’t think that that’s a roadblock.

I do believe that content is the problem. There’s two roadblocks. The first is simply creating 3-D models that look really good and expensive, and we need more people to become great 3-D modelers. The second is narrative, and there’s two challenges here. One is in general, storytelling is really hard to do well. There’s a reason why Recode has succeeded compared to everyone else because you guys know how to do storytelling. The second is, the traditional model of storytelling doesn’t work in VR. We could talk about all the reasons ...

Talk to me about a few.

The first is attention. When you have a listener to a podcast, you have her attention. There’s no other sounds going through those earphones. In VR, if you want somebody to look at a specific spot at a specific time, you can’t force that. So there can be some very important event going on. It could be a sidelong glance between two people. It can be something moving in your field of view, and the user can be looking at her feet or she can be looking up in the sky. And it’s hard to ...

Get people to focus.

VR is anarchy. People can look anywhere whenever they want.

It’s really good.

Film is fascism. It’s great. The director, if she tells you where to look, when to look, and we have the attention.

Yeah, this is what you will be doing. Yeah, that’s right. They directed you. And in VR people have choices.

And there is a lot of smart people trying to come up with solutions to ...

But you can just manipulate people. I’ve heard people are manipulable.

So you can actually force their field of view, but then you get motion sickness. So I’m a big wimp when it comes to motion sickness. If the virtual camera moves and I’m not moving with it, I get dizzy.

Right, right. All right. Talk about more problems. One is obviously these rooms where you wear the headsets, which are heavy and onerous. And I’ve had lots of arguments with people. Typically, I hate to say that it’s a man that’s like, “Oh, it’s fine.” I’m like, “It’s not.” It’s not something that average people are gonna want to do for very long, like you were talking about.

You could see you should do it in training, but it’s a singular experience. You’re alone, you feel isolated, you know you look stupid — Again, the headsets are still not ready for primetime. It feels like it could be lighter. And you see them in sci-fi in a way that ... You’re used to them like that already and you wonder why you’re wearing this giant helmet, essentially. Where is that in the process, or is it just a matter of cost in development?

So when we talk about the downsides of VR, and you hinted at one of them, and the first thing from my mind is distraction, which is you’ve got this helmet on, you can’t see the walls, you can’t see if your cat comes in.

You’re worried about hitting ...

You smash into things. I have literally saved lives in my lab from the head of the BBC, Lord Tony Hall when he did this flying demo, decided just to do a backflip.

Oh, dear.

And he’s in his ’70s, and I was right behind him, spotting him, and I caught him and he was just fine. But in general, you’re starting to see more of these accidents occur.

In the VR he was doing a backflip so he decided to physically do it?

No. In VR, he was taking off like Superman, and the way you do that is you put your hands over your head to take off like Superman would, and there’s some haptic feedback you get from the ground, and he just went with the motion and decided to jump backward.

Oh my God.

And we’re very careful in my lab, but Jeremy doesn’t come with Oculus and Jeremy doesn’t come with the HTC VIVE. And there’s some news out of Russia, which — how much we can trust in a Russian news agency is another story. About three weeks ago, a man while playing a video game in VR fell through a plate glass table and died.

Oh, wow.

And again, I can’t verify that.

Because they were moving? Because they were moving in the space.

It was a paragraph out of Newswire.

Right. But you could see that. You could see people moving. You don’t know where to go. You’re also nervous about what you’re gonna hit and the edges of where things are. So they have to be empty rooms. That’s why ... Like you’re thinking, if this is gonna be commercial, you’d have to have like a room of empty ... like a store of empty rooms where people ...

With mattresses.

Yeah.

Once a week I catch somebody. So you did the demo upstairs, we’ll talk about that. What you didn’t do are the ones that are more perceptually deranged. Things like walking a plank, or we do things that are designed to just be fun. But by definition, VR is intense. We choose to do things that you wouldn’t do in the physical world. And safety is something that I [take seriously].

Interesting. And then, we talked just briefly about haptics and other things because that still is not there, and smells. Tell me about smells, because I get sight and sound. You kind of have that nailed, essentially. It’s just in more of development, and haptics is more difficult, where it pushes back at you, or you grab something and you actually feel when you’re grabbing.

Yeah. The best way to do haptics is what we call passive haptics, and that’s what the location-based VR companies are doing, like nomadic VR up in San Rafael, and that’s a fancy word. If there’s cobwebs in VR, they hang string. And if there’s a table in VR, they put a bar.

It’s like an old radio show.

Yeah, haunted house, radio, exactly. So to do haptic using haptic devices, we have one of the heroes here at Stanford, his name is Ken Salisbury, and another here, her name is Allison Okamura, that does medical haptics, and it’s really, really hard and expensive to do haptics well, so what the companies have chosen is a little bit of haptics, for example, vibration of the controllers, and that gets you a long way.

A little bit, but it’s not real.

It’s not the same type of feedback you get from a handshake, which is, if you take one hand and shake your other hand with it and press really hard, the amount of devices it would get to get the amount of force from all the different angles is bigger than a car.

Oh, impossible. Yeah. What has to happen? What’s the breakthrough that has to happen?

If I knew that, again, I’d be a wealthy and famous man.

Yeah, because that’s really ... Feeling things is really ...

I mean, the good news psychologically, we’ve done about seven or eight studies on social haptics, meaning when you feel touch from another person and psychologically, even a tiny bit of haptic feedback really goes a long way. So I do think it’s an important cue and we should include it, but it’s nowhere near as advanced as sight and sound.

Sight and sound, and then smell.

So smell, interesting ...

Taste, eventually.

Taste, I’ve never seen any of your demos of taste yet. I don’t know how ...

You should. Why not?

Yeah.

Disney does this. Whenever you’re watching one of their movies using their glasses, they always shoot water and stuff at you.

Oh, they do?

Yeah, and smells, like all of a sudden it’s that ... cinnamon buns everywhere.

Yes. So it’s ...

Yeah, I wanna hear about these doughnuts.

So smells, the problem of how to create a novel smell by combining a certain number of primitives of chemicals, that’s pretty easy to do. In other words, if you have a set number of chemicals in the lab, you can produce a lot of smells. Now, with sight and sound, when you see an image, the image refreshes, meaning 90 times a second, you replace what was there before or what was there disappears. With sound waves, the same thing happens.

With virtual smell, if there’s a stinky bird that flies by you, you beam some scent into the nose area. And the problem is that when that bird flies away then the scent lingers. In other words, you have to have fancy fan systems to clear the scent. And now you’re starting to see some demos that are getting better, but that’s been the holdup. It’s not creating scent, but clearing it when it should.

Oh, just like you’re at the ocean, you’d wanna smell the ocean and then you wouldn’t, right?

When it’s there, it is stunning. There’s some good demos up.

Doughnuts? What was the doughnut thing?

A postdoc of mine, his name is Benji Lee, we were just about to publish papers coming out in a few weeks that asked the question, “What contributes to feelings of hunger?” And imagine you have a doughnut in your hand, okay? And you’re bringing that doughnut to your mouth and you’re about to eat it. There’s three senses that are going into that. You see the doughnut in your hand, you feel the touch of your doughnut on your skin and you smell the doughnut as it gets close.

What we did in VR is we created an experiment where you could cross these conditions, where we could basically ... you could either see the doughnut or not in your hand, you could feel a plastic doughnut in your hand or not, and we could put some doughnut smell in front of your nose, yes or no. And what we could do is we could parse the unique contribution of touch and scent to how much doughnuts you want to eat later on, to how many doughnuts you wanna eat later on.

And what we discovered in this paper — and it’s very preliminary, very preliminary small samples, so take it as the first step of many — the two competing hypotheses was that when you had this very realistic doughnut simulation that you’d wanna eat more, priming, or you wouldn’t wanna eat more, association. And after the study, we had a set number of doughnuts on the table and we allowed people to eat however many they wanted, and after touching and smelling the doughnut, they wanted to eat less.

Oh.

So it acted as association.

They didn’t get to eat the doughnut, Jeremy.

Well, they could take as many as they wanted.

It’s eating the doughnut is the key part. I get the touching and smelling there, but eating the doughnut is what ...

The big idea behind this work is imagine if a beautiful hamburger, looked like a hamburger, smelled like a hamburger, but it was really a vegan patty.

I see.

We just solved climate change. We’ve just solved the obesity epidemic. It’s just, can you give the experience of tasting amazing food ...

And not really doing it.

Yeah.

Oh, that’s interesting. You could move that to a lot of things, a lot of facts and thoughts of things. There’s a lot of experiences where you wanna have the experience but not the side effects. All right. We’re gonna talk about that when we get back. We’re here with Jeremy Bailenson. He’s the founder of the Virtual Human Interaction Lab at Stanford University where we’re taping this podcast, or digitally taping at least, and he’s a professor in Stanford’s Communications Department. He’s the author of a new book, “Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do.” When we get back, we’re gonna talk about what it could do in the future.

[ad]

We’re here with Jeremy Bailenson, a professor at Stanford who’s an expert in virtual reality, but he’s doing some really much more interesting things with this than playing a stupid game or touching a whale or something like that. Not that I mind either of those things.

But some of the stuff you just showed me was empathy and we’ve talked about that. We were on a panel recently discussing that. One of the things I agree with you, training is a great way ... and the military has been doing this forever, correct? Putting people in situations, it’s essentially role-playing but with using digital tools. Talk about the empathy part. You’re debuting this ... Talk about this project.

So the recent project I’ll talk about in a second, it’s called “1000 Cut Journey.” But as a lab, we’ve been studying VR empathy since 2003. And when I arrived at Stanford, we had a small grant from the company Cisco, and a brilliant woman named Marcia Sitosky, she said, “Jeremy, can you use VR to do diversity training? The way we do it now it’s informational, but it’s not powerful.”

And so we developed what you just did called the virtual mirror, which is you walk up to a mirror, you see your reflection, and what the neuroscientists call, body transfer occurs. Meaning, as you move your physical body, you see your virtual body moving synchronously at the same time. After about four minutes, the brain expands its schema to include that virtual body. So our big idea is you walk up to a mirror, you see yourself as someone else.

A woman or not.

I can change my change my gender, my age, my race. I can become a different species. And then you experience some trauma while wearing the body of another. You walk a mile in her shoes. And since 2003, we’ve been publishing studies that show how it can affect ageism, racism, discrimination against the disabled, pretty much any domain in which an experience of walking a mile in someone else’s shoes would help.

They truly feel that, if they truly feel what the person feels.

It’s another cue that they get that they wouldn’t have otherwise. We do tend to look at emotions, but as a lab, what sets us apart from a lot of people that do this work or that think about this is that we look at behavioral change, because with issues of race and gender, all of us say we wanna be better and most of us wanna be better, but it’s actually hard to change your behavior. And so we tend to look at outcome measures.

So pertinent now for sure. One of the things you were talking about is this idea of implicit racism. I think it’s explicit and just people don’t say it. That’s different than implicit. They just think they’re now allowed to say it, or they’re able to say it because we have a president who says it out loud. But I don’t think it went away. Even Martin Luther King was writing about that. It’s like we’re just unveiling what’s already there.

So this is an idea I got. I became a young black kid, a boy, and I got 1000 cuts. You were using the more minor things, not the more, the real heavy-duty race. It was you getting picked on unnecessarily, having people make remarks about being black casually, kids doing it to each other, the teacher digging on you for doing the same thing a white kid did, that kind of stuff.

So this is a collaboration with Courtney Cogburn. She’s a professor at Columbia University and she studies implicit racial bias. That’s what she does academically. And she and I worked together for over a year just on the storyboard, working with her team members on things that have happened to them in their lives, watching documentaries and just talking to lots of people. And what we came up with, it’s about a 10-minute journey where it was important to Courtney that the idea is these types of events happen to you every day, all throughout your life. It doesn’t happen once.

Microaggressions, I think they’re called.

These microaggressions, they happen when you’re a kid, when you’re a teenager, when you’re an adult. And in this journey, you’re wearing the body of a black person, and you start by feeling discrimination in a school room when you are in third grade, and then you are a teenager, where you have an interaction with the police that’s very different from your white friends, and then you are an adult who is going on a job interview and you’re seeing the same types of events. And so it’s about a 10-minute experience where it’s 1000 cuts, showing that these things happen all the time.

And your result is you’re trying to get people to be more empathetic. People who you would say, if you were doing that, stopping, you’d put a white police officer or any police officer in that setting to see what it’s like to be on the other side of it.

We haven’t collected data with this yet because it’s brand new, but let me tell you about the study we’re just about to publish. And this is about becoming homeless. It’s a 10-minute journey where you start out by having a home and you slowly, over time, events happen to you, you lose your job, you get evicted, you can’t afford a place to live. You try to sleep in your car, the cops arrest you from there.

And this is one we’ve studied extensively, so premiered at the Tribeca Film Festival in 2016. And since then, we’ve run thousands of people through it. And we’ve also looked, not just right afterwards but eight weeks later, and what we’re asking you to do there is to sign a petition, physically sign a petition that says, “I am willing to have my personal taxes increased to support affordable housing measures.”

And so we’re asking people to literally sign a petition. And what we’re looking at is VR compared to control conditions. Things like traditional role-playing ...

And you talk about or else you try to like stand on the street and explain why you should give something to something.

We have an informational condition as well. We have lots of control conditions. We work with my colleague at Stanford, his name is Jamil Zaki, and he’s an expert in empathy who studies the neuroscience of empathy, and he’s also not a VR evangelist. So it’s a nice collaboration because he comes at it not believing VR is gonna be better than traditional role-playing. So it’s good to have that check in balance.

Yeah. And so, one of the things I have ... Well, I think I brought it up in the panel, is that you can walk someone through what it’s like to get arrested as a black kid, say, in Baltimore. When you could curb a little bit of the terror around it, you could feel nervous in a situation you’ve never been in, but it’s a lifetime of behavior.

So I think, in this #MeToo movement, a lot of women were like, “Yeah, sure. People do that. It happens all the time.” Like they’ve got to become inured to it and aware of it in the way that men aren’t, for example. And I think putting them through one bad day is not gonna ... like really, a day that goes askew, it’s not gonna make them understand quite as much. I don’t know. I feel like it’s easy to forget that kind of thing.

This is definitely not a magic trick that’s gonna solve everything. It’s another tool that we can use. And where I think you see the most benefit of VR right now is in motivation. So in 2015 at the Tribeca Film Festival, we had a seven-minute journey, it was called the ocean acidification experience. And this was, you learned about climate change, how it affects the ocean. At Tribeca, they had this VR arcade open for about 10 hours a day for seven days straight. I had a line of sometimes 100 adults deep, waiting in line for sometimes up to an hour to learn about chemistry. And because VR is experiential, it’s novel and it’s fun, people are motivated to do it better. So leveraging this kind of phase where it’s novel, it’s a way to get people to actually experience something ...

Pay attention. You could see that. And where do you see most of the applications besides commercials, schools, training? Why go to school, Jeremy, at all? Why go to Stanford?

Well, we went down this road with MOOCs, these videotapes of professors. And so I actually work with a provost at Stanford to rethink our online education policy, and where I come in is field trips. I don’t want to replace the classroom. However, if you’re gonna go to learn about the coral reefs, why not swim around them? If you’re gonna learn about the statue of David, why looking at a 2-D picture? I mean, there seems to be some low-hanging fruits where VR actually will help. I don’t think we should blindly throw it at everything, but in those rare cases where this lesson helps, and then the cool thing about a VR simulation is, just like the digital song, once you build one, every single person on the planet will have access to it, assuming they can get the [hardware].

Right, presumably. And so let’s finish up talking about the hardware and how ... Again, it’s the purview, I’m sorry, of white guys, but you can see there, I think Mark Zuckerberg bought Oculus because he thought it was cool and he had the billions to do so. When does it become available to everybody? Because obviously cellphones were the purview of the rich and then everybody has one, and these things have a way of iterating through the society, but this is a more expensive and heavy-duty technical challenge.

So I agree with the heavy-duty technical challenge. The expense I’m not agreeing with because it costs the same as most of these video game platforms, and these have ... they’re everywhere. People have the money to buy the hardware. They do because they’re buying those video games. The reason they’re not is twofold. One, as you pointed out, to get these things working correctly, if there’s a driver update or if some of the camera ...

There’s always a glitch. I think every VR thing ... I mean, like our little handle.

Yeah, yeah, yeah. And so, because of that, it’s not there yet. The second is content. Content right now on the web is awesome. Content ... Video games as a commercial success are fantastic. And people have not come up with the, whether it’s a game or a show. They haven’t figured out how to make it worth all that drama.

Why is that? Why does it take a whole new bunch of inventing? I think it does. I think that’s the problem, is that you have technical people here in Silicon Valley and then you have Hollywood people who tell, like fascism, tell stories in a certain way. It’s a whole new genre of creative people.

I agree completely. And the one thing I will say is that, if you look at the history of film, it took us a long time to get to where we are today. So on the one hand, I’m completely agreeing that we need to break the template of, “Let’s have the film person come in and bring that over.” On the other hand ...

There’s a lot of film people who are interested.

Of course.

Jon Favreau, director of ... famous director, and he’s all in with that.

There’s a number of folks who are in. And if you think of the two industries that are grasping at VR, it’s journalism and it’s film. And why are they doing that? Because ...

We have to have new ways of getting audiences.

That’s right. The location-based VR companies, and it’s just like an arcade. They’ve got super-high-end, good tracking and a lot of these passive haptics and it feels good, like a haunted house type. It turns out that a year or two ago, how, where these things were gonna be, but now there are these places that no one really goes to anymore, they’re called movie theaters and you can just serve them right into there.

Yeah, that’s true. That’s interesting. And getting back to the empathy thing. I do think that’s the most promising, and experiential things, like I’d love to go to Bilbao, but I don’t really wanna go to Spain. You know what I mean? I’d love to walk through it, that kind of stuff, and really experience it in a different way than just looking at 2-D pictures, or hear a story in a different way that scares you. Like I could see these horror movies being really terrifying if you did them right.

But the issues around empathy and feeling, walking a mile in other people’s shoes, I mean, you don’t expect to like change people’s ... Would you put this on Donald Trump’s head and suddenly he wouldn’t insult Haitians or what? Like what’s the goal?

In the book, in chapter three, what I do is I go through, very carefully, every study that I know of that’s looked at VR and empathy, and I really take an honest approach, which is this is not gonna solve ...

And there’s not been that many studies.

There’s not been that many studies. I mean, there’s a great academic named Mel Slater in Barcelona, and then there’s my group that had been doing this. And what we’re showing is that in general it is better than controlled conditions, but it doesn’t work every time and it also depends on the content.

The question I get all the time is, “Does VR change empathy?” And my answer is, “Well, you wouldn’t say that about film or the written word. It depends on what you do.” And I’m just a hack when it comes to making VR content, right? What do I know about creating experiences? My strength is studying how these things work, and I’ve been put in the position because there’s no content out there to create these experiences like becoming homeless, and then to help Courtney work on “1000 Cut Journey.” And when smarter people than me are making the content, I think it’ll be better. But in general, to sum up the empathy research, it does tend to work better than role-playing or watching a video.

Right. I think you have to inject people with something, like drugs or something else, or some digital thing in your brain, like putting a chip in there that changes things. I don’t know.

It’s certainly not gonna help you with your use case, your oval office use case. I don’t think it can ...

I think we’re gonna give up on him on that one. And then lastly, manipulations, speaking of Donald Trump, lying, people feeling tricked. You could do that. This is like ... Just right now, the internet is getting in trouble. All the tech people are, for the results of their inventions maybe aren’t as benign as we all thought, about Russia and everything. It’s like every day, it’s another fresh horror that the tech ... the result of their inventions. This seems open to so much horrible manipulation that ... You know what I mean? Like the road it goes down.

In VR, when you’re experiencing spherical video compared to computer graphics, I think it will be different because of the expectations. With computer graphics, very few people have expectations of truth. When it comes to spherical video, that’s where we’re in this danger land because we think it’s gonna be real, but maybe it’s not. So VR suffers the same problem as all media, it can be manipulated.

Where VR comes in differently is that it’s so intense and it feels real. So the concern is not can it be manipulated more? Because the answer is yes, all media gets manipulated. The worry is that, when it’s manipulated that it creates this muscle memory for an experience that has a different result than simply reading something.

And also just on the constituents, so much attention around attention. All right, it’s stealing of attention, essentially, just recently, for example. We don’t like reality reality. Reality reality isn’t as nice. And that’s what “Ready Player One” is about to come out talking about that. They live in these horrible places and so they go into the whatever the place they go to experience a better life.

The Oasis.

The Oasis. That’s right.

So yes, in my lab we got a 20-minute rule and you’re not supposed to be in there for more than that.

Yeah, because that’s gonna work with normal people. They don’t eat too much fried food.

There’s a pair of German psychologists who published a paper in 2014 where one watched his buddy while he stayed in VR for 24 hours and took some measurements.

Oh, no. What happened? It’s like the guy who ate all those McDonald’s hamburgers.

Yeah.

Got fat and sick.

By hour 17, he was reporting not being able to discern whether events were happening in VR and outside.

Oh, dear.

So I am advocating on this show today that we should not be spending all day in VR.

Said the VR researcher.

I don’t play video games, I don’t have a Facebook account. I mean, I go outside for a living.

Yeah, you like reality reality.

I do, I do. But in the same light, we shouldn’t be ... If five years from now, listeners, you are putting on VR to read your email, then I’ve done something wrong as an advocate. I think we should reserve VR ...

No, Jeremy. There’s gonna be a chip in your eye that’s gonna be VR. Don’t you understand?

I get pitches to ...

Oh, it’s going there.

I get those pitches quite often.

Not today, we’ll be dead, but that’s where it’s going, like enhanced people.

So my advice is to go outside. Save VR for the things that make it special and we don’t need to be reading our email on VR.

Well, that was a great way to end. We’re talking to Jeremy Bailenson. He runs the Virtual Human Interaction Lab at Stanford. He’s a professor here, and he has a new book out which you should buy, “Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do.” There’s a lot of hype around virtual reality and this is like a nice, clear thing of where we’re going and where we are. So it’s not hyped or undercut. Anyway, Jeremy, it was great talking to you. Thank you for coming on the show.

Thank you so much.

---

This article originally appeared on Recode.net.