:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/52935967/GettyImages_625446502.0.jpg)
Two days before President Donald Trump was inaugurated, the New York Times ran a story about the work of two economists who published a study titled “Social Media and Fake News in the 2016 Election.” The study’s aim was to try to make sense of the “fake news” phenomenon — How are people consuming it? Is there evidence that it’s changing people’s minds? Are consumers as likely to believe false stories as they are true ones?
The answers to these questions may surprise you. Fake news stories shared on social media were important but nowhere near determinative. Only 14 percent of Americans considered social media (the fount of fake news) their dominant source of information in the months before the election. So the number of people consuming fake news was relatively small.
The authors of the study conclude that “for fake news to have changed the outcome of the election, a single fake article would need to have had the same persuasive effect as 36 television campaign ads.” Fake news mattered, in other words, but it likely had no impact on the final outcome of the election.
One of the researchers behind the study is Matthew Gentzkow, a professor of economics at Stanford University. I reached out to Gentzkow earlier this week to get a clearer picture of his findings. I wanted to know how he and his team defined fake news, how they arrived at these conclusions, and what media trends worry him the most.
Our conversation, edited for length and clarity, follows.
Sean Illing
How did you define fake news in this study?
Matthew Gentzkow
We define fake news to be stories that were circulated during the election which have been determined to be unambiguously false. Typically, stories were debunked by fact-checking websites like Snopes.com or Politifact.com.
Sean Illing
Where does the majority of fake news come from? Who or what is producing it?
Matthew Gentzkow
The majority of it is produced and posted on small websites that do not have a significant presence other than producing political stories that get shared on Facebook and other forms of social media.
Sean Illing
What was the goal of the study? What research questions were you trying to answer?
Matthew Gentzkow
Our goal was to see whether we could bring some facts to the table to sharpen the discussion about the role fake news had in the election. A lot of people have suggested or asserted that that effect was really large and potentially was enough to tip the election in favor of Donald Trump. Others, like Mark Zuckerberg, believe it’s crazy to suppose that fake news might have affected the election.
So there’s a lot of daylight between these two positions. We wanted to see whether we could narrow that down, so the question ultimately is: What was the impact of fake news on the election? We don’t answer that question because there’s one barometer that we don’t have a way of measuring, that I don’t think anybody right now has way of measuring, which is how much did seeing a single fake news story affect the way people voted.
Sean Illing
What you found is that for a fake news story to have changed the election, it needed to have been as persuasive as 36 television commercials.
Matthew Gentzkow
That’s right.
Sean Illing
I’m surprised the gap is that significant.
Matthew Gentzkow
Well, we figured out how many fake news there were total. Then we used our survey to estimate how many people saw a typical fake news story. We have a simple model of the election outcomes that lets us say, if we know how many people saw fake news stories, we can determine how much that changed the election.
So if everyone on average saw one fake news story and that one fake news story had a 0.1 percent chance of converting somebody who was a Clinton voter to be a Trump voter, we can follow that math through and say how much the vote shares in the election would have changed and whether that would have been enough to tip various states into Hillary Clinton’s column. We can map that out for different values and we can ask how big does that number have to get before Wisconsin, Michigan, and Pennsylvania all flip from one column to the other.
Sean Illing
The low persuasive rate of fake news stories suggests that they weren’t as widely read or shared as we may have thought. Is that right?
Matthew Gentzkow
Yeah, I think that’s right. People need to know just how much fake news other people actually saw. So the main thing we’re doing in the study is just trying to figure out how much people were actually exposed to fake news. To do that, we did two things. First, we put together a database of all the fake news we can find that has been flagged by those fact-checking websites. Second, we put together an online survey to estimate the extent to which people saw those stories.
If you listen to the popular discussion about fake news, the impression you get is that there’s an avalanche of fake news that crashed over the electorate during the election and so people were seeing dozens and dozens of stories every day. Once we put those pieces of the data together, you get a pretty different picture. It turns out that the average voter saw just a bit less than one fake news story over the three months prior to the election.
Sean Illing
What was the most counterintuitive finding of the study?
Matthew Gentzkow
I think you just touched on it. It’s surprising how limited the role of fake news actually was. The typical voter seeing one fake news story over the course of the election is a pretty different picture from what the public discussion of this issue might have you believe.
In addition to fake news, we also looked at how important social media was as a source of news and information during the election. If you follow the public discussion, you might get the impression that a majority of Americans were getting most or a very large share of their news from social media. Again, our results showed something pretty different.
In our survey, 14 percent of people said social media was their most important source of election news information. Fourteen percent is not nothing, but it’s certainly not the case that social media was the main place where voters got information in this election.
Sean Illing
Something you wrote in your abstract jumped out at me: “Of the known false news stories that appeared in the three months before the election, those favoring Trump were shared a total of 30 million times on Facebook, while those favoring Clinton were shared 8 million times.”
That’s an enormous gap. What does it suggest to you?
Matthew Gentzkow
The first thing it suggests is that people creating these fake news stories spend a lot more time and effort creating pro-Trump stories than pro-Clinton stories. A good possible explanation for that is that those stories are much more successful in getting shared and generating ad revenue.
I don’t think that should be surprising because real news about Donald Trump was much, much more popular and appealing to people than real news about Hillary Clinton. Television ratings for stuff about Trump, even among Clinton supporters, drew larger audiences than stories about Clinton. People just really like hearing stuff about Donald Trump. He’s an unusual, entertaining, surprising, controversial kind of character. And so in basically every domain, stories about Trump are much more popular now.
Sean Illing
I don’t want to drag you into the partisan muck here, but this seems to suggest that there are greater market incentives for pro-Trump fake news because Trump supporters, for whatever reason, are more credulous and prone to believe and share them.
Matthew Gentzkow
I think that it is certainly one hypothesis that could align with the facts. I don’t think we can say based on the information we have whether it’s that or something else. A different story, for example, is Hillary Clinton just happened to be someone about whom you could easily make up lots of stories. So it was just much easier to come up with crazy conspiracy theories about Hillary Clinton given her long experience in government and the various controversies about her email and Benghazi and all the rest of it.
For some reason, that was harder for Trump. I’m not convinced this is a great explanation, but it’s certainly possible. I think people can draw their own conclusions based on what they think is more likely. We would need more evidence before being comfortable taking a stance on differences between these two groups of voters and the way they process information.
Sean Illing
Let’s linger on the consumer part of this for a second. People talk about fake news as though it were sprung for some supply-side void, but it seems that fake news exists because enough people want it to, because there’s a demand.
What’s the role of consumers in perpetuating this phenomenon?
Matthew Gentzkow
Well, I think that the role of consumers is big, both in terms of generating fake news and in shaping the kind of media and journalism we get in general. I don’t think it’s clear that what’s going on is happening because people want fake news. One way or another people find these stories engaging and share them a lot with friends. The kind of stories you’re likely to share are surprising or potentially important.
Imagine, for example, that there’s some consumer that mistakenly thinks some reputable news story being shared on Facebook is very likely true. Then it’s not at all surprising that they’re going to share a story saying, “The Pope endorses Donald Trump” — because that’s shocking and important something people want to share.
We know the demand for this is big, and the fact that demand for it is big is an important part of why so much of it is produced. The question is: Why is the demand for it so big and how much of it is that people don’t care about the truth and are happy to share this stuff even if it’s false?
I would suspect that a meaningful amount of shares of these fake news stories are actually by people who do not believe these stories and just think they’re funny. They think they’re entertaining. They’re sharing with their friends for laughs. Part of it may be that people don’t think they’re true and like to share them anyways. Part of it may be that people are not good at judging what is true and not true. Part of it is just that anything that surprises people is likely to get shared a lot.
In the past there were a lot of norms and institutions on the supply side that helped people filter accurate from inaccurate information. I could look at the New York Times. I could look at CNN. I could look at the Washington Post. On any of those sources, I could have quite a bit of confidence that anything they report has passed some threshold test for having evidence behind it. People have debated how high those thresholds are and how accurate those sources are, but none of those major media outlets would have reported the story of the Pope endorsing Donald Trump or Hillary Clinton running a pedophilia ring out of a pizza parlor.
So it’s really a new thing about social media that those filters are gone and there’s a lot more scope for things to reach large numbers of consumers directly without any intermediary to put their reputation on the line and say, “Yes, we’re going to publish this,” and stand by the idea that this is correct.
Sean Illing
So you don’t have any evidence to suggest that people have become more ideologically rigid? Maybe it’s better to think about this as a byproduct of changes in media infrastructure and technology that make it easier for bullshit to proliferate.
Matthew Gentzkow
I think that’s right. I think that’s the main change here. There’s a separate literature on to what extent have people become more ideological, more polarized, and more partisan, and I think that there’s good evidence that they have. But that’s a fairly slow-moving and gradual kind of trend.
The difference between the way information gets distributed on Facebook and the way it gets distributed on traditional media markets is dramatically different, and there’s no doubt that that’s had an impact.
Sean Illing
Do you see fake news and filter bubbles and the broader balkanization of media as an inevitable result of the internet and social media in particular?
Matthew Gentzkow
Five years ago, I wrote a paper with my former colleague, Jesse Shapiro, that looked at the pre-social media data. We argued that the whole echo chamber phenomenon on the internet had been quite significantly overstated. It was not at all the case that the internet inevitably created the balkanization and echo chambers. Or at least that was the right description of the data we had in 2008. How much that has changed because of social media I think is a big and important question.
The most recent research that I know of suggests that maybe it hasn’t changed all that much. Maybe echo chambers and filter bubbles are still kind of overstated. I don't think we actually have a really definitive look at the data leading up to 2017.
So I think it’s open question as to how much this picture has changed since 2008.
Sean Illing
What are you more worried about: people being misled by fake news stories or the increasing tendency of people to curate their own news feeds in order to avoid stories that conflict with their biases?
Matthew Gentzkow
I think fake news is not the main thing we should be worried about. I think the broader issue of people curating what they see or what people see being curated for them is the far bigger concern. My own view — and this is not based on the data, it’s just my opinion — is that this whole fake news thing has been a little bit of a distraction. Fake news is the tip of the iceberg of a much bigger set of highly partisan stories that people were exposed to during the election. Most of them are not black and white false. They’re just very one-sided.
That’s likely the bigger issue and the thing we should be worrying about.
