If you’re a Democrat, the FiveThirtyEight forecast is probably making you feel anxious right about now.
Just last week, Nate Silver’s polls-only forecast gave Hillary Clinton an overwhelming 85 percent chance of winning. But as of Thursday morning, her odds have fallen down to 66.9 percent — suggesting that while Donald Trump is still the underdog, there’s a one-in-three shot he’ll end up the next president.
Liberals have tried to comfort themselves with the knowledge that FiveThirtyEight is an outlier among the six major forecasts, and that the other five give Trump between a 16 percent and a sub-1 percent chance of winning.
Furthermore, in a year when Clinton has long led the vast majority of polls both nationally and in contests where more than 270 electoral votes are at stake, critics have questioned the significance of the FiveThirtyEight model’s dramatic swings back and forth.
But don’t dismiss Silver’s approach out of hand, or make the lazy criticism that he’s doing it all to drive traffic to his website. (After all, his model hasn’t changed much from 2008 and 2012, when FiveThirtyEight’s forecast was remarkably stable and essentially served as a comforting security blanket for nervous liberals.)
Indeed, when you dig into FiveThirtyEight’s methodological choices, they’re perfectly defensible. The model’s reasons for its relative caution about the outcome make a whole lot of sense.
We’ll never really know whether a particular forecast was correct or incorrect, since they’re all probabilistic, and they all suggest a Clinton win is the most likely outcome. And we should keep in mind that FiveThirtyEight’s forecast is an outlier among the models. Still, here’s why you should pay attention to it in the election’s final days.
The FiveThirtyEight model has often given Trump better chances than the other models have
This week’s discrepancy in the forecasts isn’t a new phenomenon. Throughout the campaign, whenever the polls have tightened somewhat, FiveThirtyEight’s model has moved more toward Trump than its competitors have. Here’s a comparison with how the Upshot’s forecast has moved over the past few months:
You can see that while the general trend of the forecasts moves in the same direction, the magnitude of the shifts is greater for FiveThirtyEight. During the three periods when Trump has surged in polls — mid- to late July, late August to just before the first debate, and the past week — FiveThirtyEight has shown his chances being much better than the other models have.
Silver runs through some of the technical differences for his model’s different projections in this post. But overall, what’s happening now is that the polls genuinely do look different this year compared to recent presidential elections in some key ways: They have swung more often, there are more third-party and undecided voters, and Clinton has a narrow but consistent poll lead in only just enough states to win the presidency.
FiveThirtyEight and the other models have made reasonable methodological decisions that lead to different conclusions on just how important those differences are. So here are the key questions about the race that they answer differently.
How seriously should we take a new poll swing?
At this point in the race, hundreds of polls have been conducted, and the vast, vast majority of them show Hillary Clinton winning nationally and in enough states to win the presidency. But new polling is still coming in every day, and right now this new polling is tending to show a shrinking national lead for Clinton, and a mixed picture in swing states.
Every forecasting model had to decide how it will weigh those newest polls versus the older polls we’ve known of for some time. And to get a sense of how important this decision can be, try playing with the “smoothing” setting for HuffPost Pollster’s poll average charts, which controls how strongly new polls swing the average.
The “moderate smoothing” setting makes it appear that little has changed for months, and currently puts Clinton up 4.8 nationally. But the “less smoothing” setting makes the trendlines look jagged and volatile, and puts her lead at mere 0.2 points. Again, they are this different despite averaging the exact same underlying data.
(This is also a key reason why the RealClearPolitics poll averages, which drop older polls entirely to focus on the newest information, currently differ from the HuffPost Pollster averages, which incorporate newer polls only gradually in the default “moderate smoothing” setting.)
In Silver’s telling, his model moves more in response to new data partly because of a cool feature where it extrapolates how states that haven’t been polled recently have moved based on new national and regional numbers. Theoretically, that will help the model pick up on genuine new developments in the race more quickly, rather than keeping it tied down to outdated state forecasts in a changing race.
The Upshot’s Josh Katz has argued that there’s a downside to that approach. “Weighting new polls very heavily means you can be quicker to respond to new trends and to pick up turning points,” he wrote in September, “but it also makes your forecast more unstable and more likely to react to noise (what we like to call ‘chasing shiny objects’).”
So the Upshot’s model instead factors in recent poll trends more gradually. This helps filter out changes in the polls that aren’t in fact driven by genuine changes in support.
For instance, some research indicates Democratic and Republican partisans are more likely to take the time to respond to a political poll when things are going well for their respective teams. If this is a real problem this year, it may be better to take a longer view of the average. Furthermore, early voting, which has been underway in many states for weeks, could also blunt the impact of a late poll swing.
However, while caution about new polls may have been a prudent approach earlier in the campaign, we are actually quite close to the election now, so a late swing could well be real and affect the Election Day outcome. And if the non-FiveThirtyEight models are too focused on their old data and too slow to react, they might well miss such a shift, if it were to transpire.
What should we make of the higher number of undecideds and third-party voters?
Hillary Clinton is currently polling at about 45 percent nationally among likely voters, and Donald Trump is somewhere in the range of 41 to 43 percent.
But that leaves another 12 to 14 percent of likely voters who are either undecided or are currently backing third-party candidates — which is unusually high, especially this late in the game. For comparison, the undecided/third-party number at the end of the race in 2012 was just 3 to 5 percent.
Where will those undecided and third-party voters end on Election Day? Will they swing disproportionately toward a major party candidate late? Will the third-party backers stick to their guns? Will these people even bother to show up at the polls?
From Silver’s perspective, we shouldn’t be too confident that we know what they’ll end up doing. “The number of undecided and third-party voters has a strong historical correlation with both polling volatility and polling error,” he wrote in September.
This is one reason his model gets relatively more bullish on Trump when the polls tighten — if Clinton has a 2-point lead but there are so many undecided or third-party voters left, it’s easy to imagine how her lead could be overcome, depending on how those voters break. The Upshot’s model takes these factors into account too, but appears to weight them less heavily, and other models don’t account for the unusually high undecideds at all.
How safe is a small lead for Clinton, really?
Finally, one other major question about the race that gets to these forecasting differences is the increasingly relevant one of just how safe a small polling lead for Clinton truly is.
According to FiveThirtyEight’s estimate, Clinton currently has about a 3.4 percent popular vote lead. But her chances of winning are only 70 percent or higher in contests where 272 electoral votes are at stake. If she wins them all, that’s just barely enough to get her over the top.
So how likely is it that there will be either a polling error (either nationwide or in enough states to tip the scale) or a last-minute swing the polls simply don’t have time to pick up on (again, either nationwide or in enough key states)?
All the other models are essentially telling us that given the data we have, these scenarios are very unlikely to transpire — but Silver’s is warning not to count it out. After all, back in 2012, polling averages ended up underestimating Barack Obama’s national margin of victory by 2 to 3 points. So how safe is a Clinton national lead of 2 to 3 points, really?
This is in part due again to Silver’s aggressive weighting of his state forecasts to changing national and regional numbers, even when we haven’t had many new polls in particular states in a while.
To oversimplify, the other models are leaning more towards assuming that with so much polling in so many states showing Clinton narrowly ahead, it’s highly unlikely that they’ll all be wrong in the same way. But Silver’s model thinks a “miss” in national polling would likely be reflected in swing states too — even states that have been considered part of Clinton’s “firewall” up until now.
There’s sound basis in the historical data for painting the states with a broad brush, since on average, national movement does appear to show up in the swing states too. But then again, the swing states are in a different world than the rest of the country in some ways — they’re subject to millions of dollars more in ads, as well as extensive voter turnout operations. So they might not just follow along with the national trend, particularly in a year when the two campaigns are so different.
More broadly, though, Silver’s forecast is just more uncertain that the result will match what the current polling data shows (while still assuming that’s the most likely outcome). As Silver has written, his model also assumes a higher likelihood of a Clinton landslide win than many of the others. And this treatment of uncertainty better accounts for the fact that, well, we genuinely don’t know what will happen in the future.
After all, how likely was it that a report on Anthony Weiner sexting a 15-year-old girl would lead to an FBI investigation into him that would end up reviving the investigation into Clinton’s emails, and that James Comey would choose to tell the world about this just before the election?