:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/60429477/498676144.jpg.0.jpg)
On the latest episode of Too Embarrassed to Ask, Recode’s Kara Swisher talked with Northeastern University researchers Christo Wilson and David Choffnes about a popular online conspiracy theory: Smartphones are secretly turning on their microphones to record conversations and serve us more targeted ads. Wilson and Choffnes just completed a year-long investigation into 17,000 of the most popular Android apps; they concluded that we provide so much info about ourselves through other means that can explain ads that feel eerily well-targeted to our lives.
You can listen to Too Embarrassed to Ask on Apple Podcasts, Spotify, Pocket Casts, Overcast or wherever you listen to podcasts.
Kara Swisher: Today on Too Embarrassed To Ask, I’m delighted to have Christo Wilson and David Choffnes on this show. They’re both assistant professors at Northeastern University and the College of Computer and Information Science — in Boston! — and they’re leaders of a team of researchers that tied to determine whether our smartphones are secretly listening in on our conversations, which is a very popular online conspiracy theory. Christo and Dave, welcome to the show.
Dave Choffnes: Thanks for having us.
Thank you for coming. Anyway, let me just jump into questions. I read that you conducted the study at Northeastern of the course of a year. So, why don’t each of you explain what it was about, so for people who aren’t sort of following this online conspiracy theory...
Christo Wilson: Sure. So, we’ve also heard these conspiracy theories that your smartphone is secretly either watching you or listening to you, right? There’s apps that maybe activating the microphone or the camera in the background without your knowledge. And then potentially this data’s used for things like targeted advertising. There’ve been a couple-
Or worse.
CW: Yeah. So, there’s been a couple examples where this was actually true. There was a company called SilverPush that was actually using the mic to listen for TV’s in the backgrounds, to see what shows you were watching. There was a recent ... I think it was a soccer app that was also listening for TV noise in the background. But these examples aside, we don’t really know if our phones are listening to us or watching us. These cases are really anecdotal. So, we wanted to figure out at scale, you know, are apps really doing this pervasively or are these just kind of one-off bad actors?
Meaning that they put these apps in and you aren’t really clear what they were doing in the first place. So, like go to the idea of distrust in the reason why people think this is so. There’s all kinds of conspiracy theories online about lots of different things, including from aliens to alligators in the toilet, things like that. Why did this one come up, from your perspective?
CW: So, I think there’s a combination of reasons. One-
This is Dave. This is Dave?
DC: Still Christo.
CW: This Christo.
Is it Chris... Okay, Christo. Alright, okay. Good. Okay, go ahead Christo.
CW: So, on one hand, we know our devices are very powerful. They have all these capabilities, and we see that apps use them, right? When we install an app on an Android phone, one of the first things it does is it asks, “Can I access the camera? Can I access your contacts? Can I access all these things?”
Right.
CW: But, then you never hear about that stuff again, right? The app is just there, and then at some point in the future, you know, there are these weird serendipitous things that happen like you get an ad for a car that you just happened to be talking to your, you know, friend about. And the coincidence is a little bit too much. The phone is there. We know that the tracking is possible and that this targeting is happening. It’s hard not to believe that, you know, putting these things together, you’re under surveillance.
Right. Where you feel that you might be.
CW: Yeah, exactly.
DC: And just to add to that, I think one thing that comes up is we tend to think about surveillance in the way that humans do. That if somebody learned about something that you talked about with a friend that meant that they were listening to you, but I think it’s harder for most people to make the connection between how much they give away in their online activities to these companies that can target advertisements to them because it’s not just when you’re on a Facebook-
Right. They’re not aware of that.
DC: Exactly. It’s not just when you’re on Facebook looking at someone posting about Hawaii. You’re also probably being tracked by a Facebook like button when you go and visit a website about Hawaii. And so if you’re thinking about that trip to Hawaii, you probably left some breadcrumbs that that’s the case, but you weren’t aware that it was happening. So, when you see that targeted ad, all of a sudden you think, “Ah! They must’ve been listening to me”.
Right, because you also could be talking about it. You’ve already ... You’re doing a lot of online intent things that are signals, essentially.
DC: Exactly, and the other thing I’d like to point out is, for those who think that your phone is listening and that’s how they’re figuring it out, I’d invite anyone to think [about] how good the quality of the translation from our personal digital assistants is.
Right.
DC: From Siri or from Alexa. We’re not there, in terms of being able to interpret what you’re saying, especially with noise. So, you know, applying Occam’s Razor to this problem, it seems like it’s much more likely that you would expect to see this targeted tracking based on other things that you’re doing online, as opposed to the microphone listening to you translating that to text and being interpreted.
Right because it’s much more actionable. It’s not just actionable, but it’s actually “You did it, and they can see you doing it.”
CW: Yeah. It’s very direct.
Right. Exactly. So, now you’re not saying that the phone is never listening to you. As you said, there is some apps that are listening to you and there’s some cases where you give it permission to listen or take pictures or look at your photos — Snapchat, anything you use has that ability. Twitter.
CW: Correct. So, there’s many apps that are explicitly recording media. That’s their whole purpose and we expect that, so there’s no privacy violation. And there’s a few known cases where there was unexpected recording, like the SilverPush case or the soccer club case. But, the question for us was is there more unexpected recording going on then we realize?
Which is what you were looking at. Now, where did the original conspiracy, I mean in this case, come from? Where did it ... Did you find at a point of origin or you were not studying that? And how did it become so widespread?
CW: I don’t know. I mean, it wasn’t really ... That’s not really what we were looking at. It was ...
Sure.
CW: What’s going on now in the App Store versus-
Whether it’s true or not.
CW: Yeah.
Right. Right.
DC: From our perspective, I mean, look back to SilverPush and that was something that motivated our study is that there was a confirmed example of this happening. We didn’t want to just look at one example. We wanted to understand how widespread it was.
Now, before the study, did either of you believe it might be true? Did you think, “Oh. This sounds plausible,” because it sounds plausible. There’s a plausibility to it which is I think the reason why it spread so far and wide.
CW: Yeah. We totally thought it was possible. I mean, research takes a lot of time and effort. We wouldn’t have engaged in the research if we didn’t think there was something to find.
DC: Yeah, and you know we had been studying privacy from apps, and in terms of identifying when textual information about you, your GPS coordinates, your usernames, your passwords… We found tons of examples where those are mishandled, and it continues to surprise us. Although, you know, at this point, we sort of expect to be surprised, as it were. But, nonetheless, when it came to this kind of audio or video recording, sure, you know, we certainly thought that if we look at enough apps, we’re going to come across something.
Alright. Can you explain what you did, what you actually did to show this was not the case that ... Largely Google, Amazon, Facebook, all the bigs were not listening in on you per se? They are following you. Talk about sort of what you did exactly.
CW: Sure. So, we got around 17,000 Android apps. Mostly from the Play store, but also from some third-party app stores as well, and we gathered them all together, and we put them through this testing apparatus we developed. So, all of those apps got loaded on the test phones. The phones were very carefully set up, so that there was, you know, a couple pictures lying around and some music and some audio. Things for the app to find and then the app was automatically run and exercised. So, tapping on buttons, typing on the keyboard, and that would happen for around five minutes and in the background, we were recording everything the app is doing. So, anything it’s sending out over the network, we have that traced. And so then, we ran all these apps, and then we looked to see, did we ever observe a picture or an audio clip or video file gets sent to someone? And when we did, then we would trace it back. So, was that explicit? Like, it was a camera app and you clicked the shutter button.
Snapchat, and you wanted it to happen.
CW: Yeah. Exactly. The Snapchats. Or was this unexpected? You know, an app that you give it permission to use the camera, but you never clicked a shutter button or anything like that. It just took a photo or it found a photo on the storage and it just uploaded it without your knowledge.
Right. But, this was only on Android. Why was that?
CW: So, iPhones, unfortunately, are just much harder to work with, much harder to set up these kind of automated testing apparatuses at scale and much, much harder to collect all the apps necessary to kind of feed the experiments.
Right. And did you find one more protected than the other? Does Apple have more protections in it or privacy protections for its users?
CW: So-
Is that why?
DC: Yeah. I wouldn’t say that one OS is necessarily better than another, on that front. For example, when we did find cases of suspicious behavior, we know that the companies make apps for both platforms. So, I don’t expect that there’s necessarily less privacy risk on one platform versus another. Just in terms of being able to do our experiments at scale, Android was the only thing that we could feasibly do.
And who did people think were listening the most? Was it Google or Facebook or who did they think was doing just the phone itself, because the phone is an Apple phone for instance. So, is it Apple they’re nervous about or is there just an amorphous someone’s listening?
CW: It seems to be amorphous. It’s this unknown third party omniscient advertising-targeting entity. The advertising ecosystem is actually very complex. We, as researchers, have a hard time fully understanding it. Ordinary people have no ... They have no visibility. There’s no way to know.
So there wasn’t anyone. They weren’t saying, “Ah, Google’s listening in on me, or Alexa, or whoever?”
DC: I mean, I’m sure that they said that. I’m sure that they said a lot of other names, as well.
Yeah. But you didn’t notice it in the studies. It is an amorphous thing that people are like, “They’re listening.” I’m like, “Who? And why?” “Advertisers.” “Well, which ones? Why? Who’s the overall body that’s conducting it? And would Apple allow this? Would Google, Android allow this to happen? And wouldn’t they want a part of it?” I’m like trying to carry it out to the larger extreme. So when you came out with this, what was the reaction?
CW: I’d say-
People don’t believe you. Right?
CW: Yeah. I’d say it was sort of mixed. On one hand, there’s the optimistic outlook, which is, we looked at all these apps and we found very few cases of unexpected media leaks. So in one sense, this fear is sort of unwarranted. But on the other hand, we did find a few cases where there were apps doing things that were very privacy invasive and unexpected, so it’s not like the number is zero. It’s not. So it’s glass half full, glass half empty.
Right. Right. But in general, the overall idea that it’s constantly monitoring you is not ... It’s that you’re giving it signals more than anything else. Have developers changed anything since your report came out? Or did they have to?
CW: Yes. So there were a bunch of apps using a particular library that was allowing them to record the screen. So when you are in one of these apps, everything would just get recorded and sent to a third party. So we responsibly disclosed that. In the paper, we talk about an app called GoPuff. GoPuff has since removed that library. And it looks like a bunch of other apps that included this particular library, have also removed it.
And this is ... Explain that. They use it for what purposes?
CW: This is a library from a company called Appsee, and it’s meant for developers to help debug the application. So if someone says they’re having a problem with the app, they can’t figure it out, there’s actually videos of them using the app. Or you can just use it sort of generically, like-
So they’re surveilling it.
CW: Yeah, it’s surveillance.
Yeah, so they’re surveilling you.
CW: Yep.
Right. Essentially.
DC: You can think of it as, if you had a problem with your computer and you call tech support, the guy from tech support is probably going to show up, look over your shoulder, as you do the thing that caused a problem. They’re doing this, except they’re doing it remotely.
My colleague, Jason Del Rey, is one of the people who said he thought Facebook was spying on him. Great, one of my employees. Last year he tweeted, “There is nothing you can do to convince me that Facebook isn’t using my phone’s mic to target advertising at me.” Jason. So what would you say to convince him?
CW: Well, I guess, Facebook hasn’t done themselves any favors by filing patents for exactly the technology that Jason is describing.
Yeah.
CW: But as Dave said earlier-
And their privacy snafus with the Russians. I mean, I think, at this point, only Donald Trump believes the Russians weren’t involved in all this spying, but OK.
CW: Of course, yeah. Facebook is making it too easy to believe this about them. But I guess the counter-argument is really that you already give Facebook such a huge amount of information in terms of the things you view and you click on. They can see your browsing history, based on their like buttons. You’re posting messages full of rich content. All of that stuff is easy to mine and use for ad targeting. It would be much more difficult for them to be recording this audio, and then transcribing it and trying to analyze it. You would notice the drain on your battery, all this network traffic that’s being sent. It would be hard to hide and it just doesn’t seem like it would be worth the cost.
DC: And just to clarify, Facebook is spying on us. They’re just doing it without using the mic. They get a much richer view by how we interact with Facebook, how we like things, how we click on links, how we browse the internet. So I think that’s the main-
So, you’re doing the work for them.
DC: Exactly. That’s the main issue is it might be a little [crosstalk 00:17:09].
But talk about this patent and what they want to do because they’re coming out with a device, a screen that will be in your home as an IoT device, just like Amazon Echo and Alexa. The Amazon Echo on Alexa and Google’s Home. Sorry.
CW: Yeah. So those devices are supposed to only be active when the hotwords are used. But again, of course, there was the example of the Google Home Mini, which was just recording all the time.
Right.
CW: So this is another thing that we are actively studying, is these IoT devices.
Right. We’re going to get to that in a second from some questions. But in terms of what Facebook’s coming out with, they want to be listening to things if you, presumably, give them your permission.
CW: Yeah, so it’s unclear ... Well, I’m sure they will eventually use this for ad targeting, but at least initially, this is just a way of trying to go against Amazon, or I guess Google, at this point, because they’re selling more home devices. It’s just another device, another platform that they have to control.
All right, another question. What have you heard from people who believe in this conspiracy theory, since your study came out? Do they still not believe you, or do they continue to believe their conspiracy theory? Have they argued with you about it?
DC: I don’t know that we’ve engaged in any particular online arguments. I avoid them like the plague, personally. But nonetheless-
I jump right in.
DC: No one has engaged me yet. And at the same time, we do want to be clear that we looked at 17,000 apps. We didn’t look at every app. We interacted with them automatically for five minutes. We didn’t interact with them forever. We didn’t try everything in the app, so it’s still possible that some of these apps are spying on you in ways that people fear. It’s just that we don’t have any evidence of it. And we looked at the most popular apps. Thousands of the most popular apps, so it’s very likely that the apps that are on your device are not doing this kind of behavior. At the same time, it’s not impossible.
Right. And did the app platform developers that you said from some who had been abusing it, but what about Apple and Google who are the principal phone makers?
CW: Yeah. So we reported this to Google. Google escalated it to their privacy team. We know their privacy team was in contact with many of these app developers to try and clean up the behavior. And Google issued a statement saying that they had taken remediative action.
DC: “Appropriate action.”
CW: “Appropriate action.” So we don’t know exactly what they did. They seem to have done something. It might be nice if they would take a stronger policy stance against some of these things, but they did take our report seriously.
DC: Mm-hmm.
Okay. And we got a question from one of our readers, Scott Weil, “Has Intel ever put technology into their chips to collect data about how a computer user computes? Have they ever collected data on clicks, keyboard usage, or program usage?”
CW: Not that we know, but-
DC: But we haven’t looked at that.
CW: Yeah.
DC: Great question. Not really the focus of what we’ve looked at so far.
Okay. All right. Have you or your colleagues looked into other forms of digital surveillance? For example, whether Alexa or Google Home are listening when they’re not awake? Is this the next area you’re going to look at? These IoT devices? And talk a little bit about those.
CW: Yeah, sure. So we actually have a state-of-the-art lab set up like a studio apartment here at Northeastern. It has fully functional internet-connected fridge, washer, dryer, dishwasher, TV, cameras, video doorbell. Just sort of everything you would expect to see in a really smart home. All of these devices are connected to a router where we have full control over all of the network traffic, so we can monitor it. And we also can correlate activities that we do with these IoT devices. You know, how we use the devices versus what they’re sending over the internet.
So this is an ongoing study. We don’t have anything to report publicly now, but certainly, we have several Amazon devices and Google Home devices, so there’s many devices. Even the fridge has a microphone for reasons — we’re not entirely sure why that was necessary. But these are all things that we’re very interested in understanding. Are your devices listening when they shouldn’t? Besides when you say something that sounds like the hotword, but it actually isn’t, and then it starts listening. And when these devices aren’t supposed to be listening, and they’re still using the internet connection, what are they using it for? Why are these devices so chatty when, in theory, they shouldn’t need to be?
They should be silent. They should be silent. So I think one of the things is there’s an opportunity for abuse that’s accident-based, too. Is that it’s just that there’s a glitch and it listens or that it’s on, and why doesn’t it listen? Or someone could hack into it and it listens. Correct? That’s the danger with all these IoT devices is that they can be ... You know, you’ve seen a million movies where they turn on the camera on your computer, or on your phone, or things like that.
DC: Absolutely. That’s something that we’re trying to understand as well. It all comes down to trying to understand what’s normal behavior for one of these IoT devices, and then what deviates from that behavior in terms of what we see on the network. This is something that we’re actively working on, but there’s a lot of challenges here. There’s just so many different kinds of devices and they each have such different purposes. It’s something that we’re certainly well on our way to getting a better understanding of, but it’s a hard problem, in general, just because there’s thousands of these devices and we can’t just load them onto a phone like you can do with the study that we did with apps.
Right, exactly.
DC: That’s why we need this lab.
What are you thoughts on this surveillance society? I think there’s a great article in the New York Times over the weekend about AI and facial recognition. Obviously, there’s cameras everywhere in China, and there’s more and more cameras here. They all upload to the cloud and do all kinds of different data... We are in an era of increasing surveillance of ourselves and information that we give out freely to people, correct?
DC: Yeah, absolutely.
I mean, there’s no beating that.
DC: Oh yeah. We have more surveillance now than at any point in human history.
It’s not a surprise that people think this conspiracy theory that is listening to you. Everything is listening to you in a weird way. It’s kind of worse than that, it’s right out in the open essentially.
DC: Yeah, if you generalize the word from recording my audio to be a little bit more general, like is it recording my keystrokes? Probably. Everywhere you visit? Absolutely. Everywhere you’ve been? Yes.
CW: To expand it, it’s not just, is it listening to you, or is it using your camera, but is it watching everything you’re doing? This includes recording the screen while you’re on the phone, but also, there are smart TVs that are watching what you’re watching and extracting data about that and sending it over the internet.
Right, so what advice do you give to people who are worried about this? I think it’s normally a normal thing to worry about. I think it’s a very reasonable thing to worry about.
CW: If your specific worry is, “My phone is engaging the camera or the microphone without my knowledge,” at least for now, that’s not a prominent threat. Maybe in the future that will change, but at least for now, that’s not what you should be worried about. You should be worried about the routine recording of browsing history, app usage, GPS. Those are absolutely happening and it’s omnipresent. There are things you can do like changing privacy settings, installing ad blockers and tracker blockers. Those are the prime threats, and there are mitigations.
What should people do if they are reasonably worried? What are some of the things they can do? I cover my cameras, all of my cameras on all my devices because I just assume that at some point even accidentally, they can be turned on.
CW: That’s a good start. If you use major services like Google, they have fairly detailed privacy preferences where you can go and turn off, say, location history, search history, browsing history, just “don’t record this stuff.” Taking it a step further, you should have at least an ad blocker, possibly more in your browser, both on your desktop and your phone. If you use Firefox on your phone, you can install extensions into it, including ad block. Same thing with iOS. You can download apps from the store that will implement blocking. You should do that.
DC: Just sort of more general advice for those who might feel overwhelmed and feel like throwing their phones in the lake: In terms of just practical advice, think twice about an app before you install it. Do you really need that app? Why is that app free? What data might it be collecting and is it worth it?
Right.
DC: Same thing for internet of things devices. Why does this thing have an internet connection, and do you need the things that that device advertises as the reasons why it does? If you don’t, you’re probably better off just buying a model that doesn’t have that connection, and that’s one less opportunity for your information to be exposed. It’s more about just sort of limiting the attack surface, as opposed to deciding to just cover everything in tinfoil.
Right, right, well that too, works I have to say. I’ve been wearing a tinfoil hat for many years now. Actually one of my first jobs at the Washington Post I worked nights because I was the low person on the totem pole. Every single night, someone who called me and was wearing tinfoil on their head because they were being attacked by… things, and I actually believed them for much of my early life.
It was a really interesting thing, because this thing has persisted for a long time, this idea of surveillance. A lot of it is true, and a lot of it isn’t true. It’s really interesting. I’m glad you guys did this. So what’s your next thing you’re doing?
DC: In terms of this study, I think the main next step is to figure out what those IOT devices in the home are doing. Are your cameras watching you when they shouldn’t? Are your listening devices listening when they shouldn’t? Is your TV surreptitiously recording what you’re watching and sending it to other parties? What is your fridge doing with that microphone? We’re trying to answer these questions in this much more challenging environment of home IOT.
Mm-hmm, and then from there?
DC: It’s not good enough from our perspective to just tell everyone the bad news. That certainly doesn’t bring much comfort to anybody. We’re also interested in developing solutions that allow consumers to regain some control over their privacy. One of the advantages of our approach is that we just look at internet traffic. If we can reliably tell that the internet traffic contains something you don’t want, we’re no longer beholden to what Android or iOS supports, or what an app’s privacy settings are, we can just block that network traffic.
That’s the kind of service that we’d like to provide. This is something we could even provide as a service that you install on your home router, or something that you plug into your home router. That’s really, it’s not enough, I mean we start with the bad news, because we have to figure out what the problem is first, but we’re always moving towards giving people good news, how you can address this problem, how you can move forward to a place where you have better controls over your privacy.
Bur right now, the phone is not listening to you to do ads, except when it is. Except for those bad apps, correct? You can safely say for most of the apps?
DC: Right, that’s what we’ve seen so far.
Okay, alright. This has been super helpful. We’ve been talking with Christo Wilson and David Choffnes. They’re two professors from Northeastern University, and they are debunking the idea that our phones are listening to us, except when they are. We really appreciate you for coming on the show, and thank you so much for talking to me.
DC: Thanks for having us.
CW: Thank you.
This article originally appeared on Recode.net.
