Polls can be wrong

With the election two and a half weeks away, the conventional wisdom on several closely watched races is settling in. The political world thinks that Democrats are likely to take the House, that Beto O’Rourke (D) is doomed in Texas, that Joe Manchin (D) has locked things up in West Virginia, and that Marsha Blackburn (R) has pulled ahead in Tennessee.

When the votes are counted on November 6, all of this could turn out to be spot-on — or some of it could be dramatically wrong.

As polls come in during the final days of the election, we should keep in mind two reasons why what they’re showing now might not necessarily be what we see on Election Day.

First, there’s the possibility that the polls could shift in these final days. This could be due to new developments in individual races, like scandals breaking or candidates ramping up ad spending. A change in national news headlines could also benefit one party or the other generally. Remember, it was just a week and a half before the election that FBI Director James Comey disclosed that the FBI had found new Clinton emails. Analysts think this played a major role in swinging the polls, and Nate Silver later concluded it “probably cost Clinton the election.”

Second, there’s the possibility that the polls could just be off, or missing something. Pollsters could be wrongly modeling turnout, to the benefit of one party and detriment of the other. Undecided voters making up their minds at the very last minute could also break disproportionately to one candidate or side, either nationally or in particular races or regions. Pollsters famously underestimated Donald Trump’s support in the Midwest by several points in 2016, and it was only a year ago that Virginia polls underestimated Democrat Ralph Northam’s margin of victory by more than 5.5 percentage points.

The best, most reliable indicator for what’s going on in campaigns is to still to look at polling averages. But we also shouldn’t overstate how reliable these polls are — especially while we’re still a few weeks out from the election. Recent history should remind us that a whole lot can change very quickly.

There’s still more than two weeks in which the polls can move

Most people remember how the presidential polls were “wrong” in 2016, but many have forgotten that national polls really did show a dramatic tightening in those final three weeks before it.

On October 18, 2016, many thought Hillary Clinton had the presidential race wrapped up. She led in national polls by an average of 7.1 percentage points, according to RealClearPolitics. Such a margin meant she’d likely cruise to victory in the Electoral College.

Then things took a turn, particularly after Comey’s October 28 letter. By October 31, Clinton’s poll lead was down to 3 points. By November 5, it was just 1.5 points. By Election Day, when polling wrapped up, it had trended upward a bit — her final lead in RCP’s average was 3.2 percentage points. (Her actual popular vote win turned out to be by 2.1 points.)

So did many state polls; for instance, between October 21 and the final pre-election averages, Clinton’s lead in Michigan shrank by 8 points, and her leads in Florida and Pennsylvania shrank by 4 points each.

This may have been in part because Republican-leaning voters “came home” to Trump. But major news developments that were unfavorable to Clinton, like the Comey letter, likely played a role too.

All this should be a useful reminder that the future’s uncertain. Two weeks ago, all the media could talk about was the Brett Kavanaugh confirmation battle, and red-state Republicans crowed that it was playing out to their advantage. Since then, though, coverage has moved on to other topics — the stock market’s volatility, allegations that the Saudi Arabian government murdered a journalist, a caravan of Honduran migrants moving toward the border.

We don’t know what will lie ahead or how it will make polls swing in these final weeks — but we certainly know such a swing is possible, in either direction.

The polls are often off by a few points, and sometimes by even more

Some of the polling misses in 2016 were infamous. Trump won the state of Wisconsin by 0.7 percentage points, despite never having led there in a single poll tracked by RCP (Clinton led the final average by 6.5 points). Pennsylvania polls missed by about 3 points, and Michigan’s by nearly 4. We remember all these because the polling error was big enough to change the outcome — from a Clinton win to a Trump win.

Less remarked upon was the polling “miss” related to the highest-profile election in November 2017 — the Virginia governor’s race. Here’s how it ended up looking on RealClearPolitics:

RealClearPolitics

Yes, the polls correctly called Democrat Ralph Northam as the winner. But they were significantly off about how much he’d win by — they understated his margin of victory by 5.6 percentage points.

Still, that was better than the (relatively few) Virginia Senate polls in 2014. Sen. Mark Warner (D) was up in the polling averages by 9.7 points but only won by 0.8 percent — about a 9-point miss.

RealClearPolitics

That’s an uncommonly big miss. But it’s quite common for the final polling averages to be off by a few points for any particular race. Just look at what happened with polling averages in some of the highest-profile Trump-era special elections:

• Georgia’s Sixth District: Underestimated the Republican’s margin by 3.6 percentage points
• Alabama’s Senate race: Underestimated the Democrat’s margin by 3.7 percentage points
• Pennsylvania’s 18th District: Overestimated the Democrat’s margin by 1.8 points
• Arizona’s Eighth District: Overestimated the Republican’s margin by 2.8 points

In all these contests, the polls were in the neighborhood of the final outcome. But they were off in the low single digits in all of them — and by more than that in the Virginia governor’s race.

That’s, then, what polls are generally good for — getting us in the neighborhood of the outcome, most of the time. But we shouldn’t exaggerate their precision or how much certainty they offer. Election night will likely bring some surprises.

We have a request

Vox's journalism is free because we believe that everyone deserves to understand the world that they live in. That kind of knowledge helps create better citizens, neighbors, friends, parents, consumers and stewards of this planet. In short, understanding benefits everyone. You can join in on this mission by making a financial gift to Vox today. Reader support helps keep our work free, for everyone. Will you join us?

One-Time Monthly Annual

$95/year

$120/year

$250/year

$350/year

Other

$

Yes, I'll give $120/year

Yes, I'll give $120/year

We accept credit card, Apple Pay, and Google Pay. You can also contribute via