When pollsters try to figure out who is winning an election, they use something called a “likely voter screen.” It’s exactly what it sounds like — a way to sort who will actually show up on Election Day from who merely says they prefer one candidate but will probably stay home.
Pollsters have different ways of building that screen. But it normally includes asking voters questions, including — beyond a direct “Do you intend to vote?” — whether they’ve voted before and if they’re registered members of a party. That’s because not everyone who says she plans to vote really will.
This likely voter screen may be one of the biggest reasons pollsters and election forecasters almost uniformly missed Donald Trump’s stunning victory Tuesday night, according to Matt Dickinson, a political scientist at Middlebury College.
Trump drew many new believers into the political process for the first time; the likely voter screens appear to have assumed that they would not actually show up to vote on Tuesday. As a result, polls with strong likely voter screens may have underestimated his strength.
“If you threw out all the Trump supporters who had never voted before,” Dickinson says — as pollsters tend to do — “you’d end up with a lot less Trump support in your polling data.”
Trump’s victory was missed by basically everybody in the polling industry. And that sets up a question: How did the pollsters so badly whiff on an election with such high stakes?
The dust is still settling, but I talked to seven experts in political science and polling for their responses.
Here’s what I learned.
Let’s not get carried away with how bad the predictions were
Before dismissing the whole polling industry as useless, Kathy Frankovic of YouGov stressed that there’s been some exaggeration about just how bad the predictions were for last night.
“It’s not a precise science, and it’s never been a precise science,” she says.
Even last night, some polling firms did reasonably well when projecting the national vote totals. Many of the final national polls had Trump up by 1 or 2 points, and several others had Hillary Clinton up by only a few points. Given that Clinton won the popular vote, it actually looks like the national popular vote projections weren’t off by that much.
Still, the real miss was that many of the state-by-state polls were badly off — like in Michigan, Wisconsin, and Pennsylvania, where Clinton was expected to win but lost. But even here, her lead was probably always a bit exaggerated by the media. The RealClearPolitics average for Clinton in Pennsylvania, for instance, only had her up by 1.5 points — well within the margin of her error in her loss. And one big problem is that the media and the public are so desperate to know who is winning that they put too much faith in pollsters who are themselves warning about uncertainty, Frankovic said.
“Journalists and everybody else often put so much confidence in the poll, and the question we have for them is, ‘Why do you do that when we tell you that there’s going to be a lot of room for error?’” Frankovic says. “And then when there is error, they turn around and say, ‘Why did you get that wrong?’”
But why were the Midwestern states so badly polled?
A week before the election, Matt Grossmann, a political scientist at Michigan State University, came out with a poll of the state that stunned the media.
Clinton, it predicted, would win Michigan by 19 points.
Clinton lost the state. Grossmann wasn’t alone in badly missing it — most polls had her up by around 4 points in the state.
The whole region was something of a black hole for pollsters. In the home stretch, Clinton was projected to be up by around 7 points in Wisconsin. She lost it by a point. In Iowa, she was only down by 3 points in most polls. She lost it by a whopping 9 points. She was winning Minnesota by as many in 10 points in some of the late polls. She won it by about 1 point.
The odd thing is that the state polling was actually quite good throughout most of the country. In most blue states across the country, Clinton did about as well as predicted.
But that doesn’t make the big miss in the Midwest any less inexplicable.
“It’s a funky thing. What makes this so weird is that the bad polling was really in just a handful of Midwestern battleground states,” says John Sides, a political scientist at George Washington University. “It’s that these effects were particularly acute in some of these Midwest states, and I’m not sure we have an answer yet for why.”
In an interview, Grossmann pointed to two big things that his team appears to have missed: 1) They badly underestimated the number of white voters without a college education who would turn out at the polls on Election Day, and 2) Republicans dubious about Trump rallied to support their party’s nominee in far greater numbers than projected.
“So it was a very different electorate than we expected,” Grossmann says. “And Republican-leaning women and conservatives who hadn’t made up their minds almost all ended up voting for Trump.”
Additionally, late-breaking voters — those who didn’t make up their minds until late in the campaign — appear to have backed Trump and would have been missed in earlier polling, according to Alan Abramowitz, a political scientist at Emory University.
One possible explanation: Polling response rates are plummeting
Twenty years ago, polling firms got about one-third of respondents on the phone when trying to figure out the state of the race.
That number has been on a slow and steady decline. Now pollsters are lucky to get a response rate that is in the single digits, Dave Hopkins, a political scientist at Boston College, points out. Because people don’t use or pick up landlines, and those who do are simply less likely to answer questions about their presidential candidates, pollsters are now dealing with a smaller pool than they once were.
“We’re getting to a point where you may just not be able to reach a certain segment of the electorate,” Hopkins says.
He says this may have been the case before this election, but that the segment being missed has never before so heavily leaned toward one candidate — making the phenomenon hard to disaggregate from the overall trend.
Of course, the fact that response rates have gone down hasn’t stopped pollsters from trying to make projections. But instead they are now doing demographic weighting — essentially, trying to map their poll results onto what they imagine the overall electorate will look like. And that sometimes blows up spectacularly in their faces.
“You’re making more and more assumptions,” Hopkins says, “assumptions that are sometimes correct and sometimes aren’t.”
Turnout across the board was way lower than expected
Before the election, Reuters polling director Julia Clark and her team made a chart. It projected that Clinton would win easily if 60 percent of the country turned out to vote, and be handily defeated if just 50 percent of the country materialized.
Reuters ultimately projected that about 58 percent of the country would vote and that Clinton would win the popular vote by 3 points. Only 55 percent of the country showed up — and Clinton only won the popular vote by around 1 point. That wasn’t enough to put her over the top in the Electoral College. Democrats, Clark says, had expected much higher turnout in urban centers.
“When fewer people vote, Republicans do better. That’s a modern political fact,” Clark says.