:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/51705297/490420682.0.jpg)
In normal times, people don’t get animated about polling methodology. But this weekend is not a normal time. And so on Saturday, the internet was treated to a bitter Twitter feud between a prominent polling analyst and a Washington journalist: FiveThirtyEight’s Nate Silver and the Huffington Post’s Ryan Grim.
Silver’s polling model currently gives Hillary Clinton only a 65 percent chance of winning the White House, which makes it the most pessimistic of the major polling models. The Huffington Post, in contrast, gives Clinton an astronomical 98 percent chance of winning.
On Saturday, Grim penned an article savaging Silver’s methodology and comparing it to “unskewing” websites that showed Mitt Romney ahead in the 2012 presidential election. Those are fighting words in the polling world, since these adjustments turned out to be totally wrong — President Obama ultimately won re-election by an even wider margin than polling had predicted.
Grim claimed that Silver is doing something similar, making adjustments to polling results that are “merely political punditry dressed up as sophisticated mathematical modeling.”
That inspired a heated response from Silver:
This article is so fucking idiotic and irresponsible. https://t.co/VNp02CvxlI
— Nate Silver (@NateSilver538) November 5, 2016
The reason we adjust polls for the national trend is because **that's what works best emperically**. It's not a subjective assumption.
— Nate Silver (@NateSilver538) November 5, 2016
It's wrong to show Clinton with a 6-point lead (as per HuffPo) when **almost no national poll shows that**. Doesn't reflect the data.
— Nate Silver (@NateSilver538) November 5, 2016
The problem is that we're doing this in a world where people—like @ryangrim—don't actually give a shit about evidence and proof.
— Nate Silver (@NateSilver538) November 5, 2016
The big dispute is about “trend line adjustments”
The centerpiece of Grim’s case against Silver’s model is something called trend line adjustment. Silver is trying to solve a problem that all poll averages face: what to do with older polls, especially in the midst of a volatile race.
For example, we know that the race has tightened significantly since mid-October. So when computing poll averages, it would be silly to simply throw 3-week-old polls into the mix as though they’re as relevant as the latest polls.
There are a few different ways to deal with this problem. One option is to just throw out older polls, the approach favored by the RealClearPolitics polling average.
Another option — the one chosen by FiveThirtyEight — is to try to adjust the past poll results to reflect the overall national trend. One advantage of this approach is that some states are polled infrequently, so projecting older polls forward can provide an educated guess about what the state would look like if it were polled today.
As Silver explains it: “if Trump led in a North Carolina poll by 1 percentage point in June, but the trend line shows him having gained 3 percentage points nationally since then, the model will treat the poll as showing him up by 4 percentage points.”
The trend line is calculated by comparing different editions of the same poll. For example, Silver explains, “if Clinton is at 46 percent in the Quinnipiac poll of Florida in August and was at 43 percent in the same poll in July, that suggests she’s gained 3 percentage points.”
Grim’s crack about trend line adjustment being “merely political punditry dressed up as sophisticated mathematical modeling” is unfair. Silver’s model is adjusting older polls in a pro-Trump direction because newer polls have shown a closer race.
If polls had moved toward Clinton instead of Trump in the last few weeks, Silver’s trend line adjustment would have given older polls a pro-Clinton bump. It has nothing to do with “punditry.”
Silver also argues that the trend line adjustment is a bit of a red herring in explaining the difference between Silver’s 65 percent chance of a Clinton win and the Huffington Post’s 98 percent probability.
@dsquareddigest @delong @Econodel @mattyglesias: The differences are 80-90% because of variance, 10-20% trend.
— Nate Silver (@NateSilver538) November 6, 2016
Silver’s model assumes that there’s a wider range of possibilities than the Huffington Post model does. One reason for this, Silver says, is that the FiveThirtyEight model assumes greater correlation among state results, so that if Trump surges in one state he’s likely to surge in others too.
