:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/51898719/shutterstock_449742301.0.jpeg)
Ian Warren is sometimes referred to as the “British Nate Silver,” having made a lucrative career out of correctly forecasting election outcomes. And like a lot of people in the business of polling and election prediction, Warren was left feeling distressed by Hillary Clinton’s upset.
“What the fuck just happened?” he wrote on his blog Election Data. “First, the near-miss on Scottish independence, then the [UK] 2015 general election, Jeremy Corbyn’s election as Labour leader, Brexit, Corbyn’s re-election and now Donald Trump’s victory.”
Forecasters like him rely heavily on data, and lots of it. And yet Warren says that in the lead-up to the Brexit referendum, “It was very clear … that no matter what I punched into the model that I was working with, numbers coming back just didn’t match up to what I was feeling in the country.”
Data analysts like Warren usually don’t trust gut feelings. But it may actually be time to turn more feelings into data. His new mission is to study up on behavioral science and use its research methods to make election predictions work again.
The following conversation — conducted over the phone and some follow-up questions over email — has been edited for length and clarity.
Brian Resnick
In your blog posts, you sound pretty despondent. Do you feel like data has failed you and your colleagues?
Ian Warren
My whole political life has been using data. I do predictive modeling. I do demographic analysis. I work for national campaigns, local campaigns. And everything I do, up to this point, has been using numbers, maps, models, statistics, data.
So last year the referendum over here, the Brexit referendum, changed my thinking a little bit about the power of analytics and data.
And the US election has just confirmed this suspicion that I’ve had for a little while that data is not really getting us the complete picture.
Brian Resnick
So where do campaigns and pollsters fail in obtaining this “complete picture”?
Ian Warren
My starting point is with political parties and their analytical teams. They currently include what we know about a person in demographic terms — age, gender, etc. — and their voting history. They use those to identify who they target. They don't include the psychological/behavioral traits of the voters yet.
[Psychological data] is just not being collected … by any of the mainstream political parties — both in the US and in the UK.
In fact, they don’t get anywhere near telling you how that person is feeling and thinking about an election. So none of these emotional and behavioral psychological data points have been collected.
Brian Resnick
But political scientists and political psychologists have been collecting that sort of emotional and psychological data for years. Reports are published all the time. For instance, just before the election, there was a study on how anxiety about demographic change could influence support for Trump.
Ian Warren
My view is that pollsters have ignored psychological research, and mostly for very good reasons. For many of the bigger pollsters, their political arm is a relatively small part of their portfolio. [A lot of the pollsters will do market research for industry clients.] So in the absence of evidence to the contrary, they've operated very well for many years with the models they have always used.
Brian Resnick
What type of questions have pollsters ignored? Can you give an example?
Ian Warren
In the UK [Brexit] referendum, I did a very straightforward questionnaire survey of people in the UK. It was purely about trying to find out their psychological makeup, their behavioral characteristics, and what messages worked with them. What I found was that the people who are most likely to vote for leave were neurotic individuals.
I should emphasize here that I am not a behavioral psychologist; I don’t have a psychology background. So a lot of this is new to me; I'm learning as I go here.
But if you look at the research into neurotic archetypes, they are extremely sensitive to external threats on a number of different levels. And immigration is one example of a threat which they are extremely sensitive to.
It doesn’t have to be a real threat; there don’t have to be hundreds of immigrants coming in to where they live. It can be a perceived threat. But their sensitivity to it is so strong that it overrides every other decision they make in life about politics and about the referendum in particular.
Brian Resnick
What lesson did the psychology survey teach you?
Ian Warren
The [traditional demographic-based] model said one thing, the survey seemed to be saying another. Not universally, but with sufficient wiggle room to say there was potential for a miss, especially in such a close contest. That was when I sat back and did some thinking. Seemed clear to me that people were making emotional decisions sometimes at odds with what a traditional model would predict.
And at the moment all I can tell you is the predictive likelihood of that person voting for X, Y, and Z based on the demographic characteristics. I want to include the behavioral and psychological traits [personality traits like neuroticism, openness, etc.] to see if it makes a difference.
And then what I intend to do is just keep going back to people. If an issue comes to us one day, then I want to see what they feel about that issue emotionally. If they’ve even heard about the issue, if it’s important to them or not. I hope that I will get a picture of which issues resonate, which ones don’t.
Brian Resnick
And campaigns could use that type of psychological data to make better strategic decisions?
Ian Warren
My instinct would be that including that data point in the model would have two effects: a) change the segments targets [of the campaigns] and, b) impact the messaging required to hit those targets.
At least that's my hypothesis.
Brian Resnick
Okay, so that’s on the political strategy side of things. What about in predicting election outcomes? How could behavioral data help there?
Ian Warren
What I'm intending to do is to segment the population into emotional and psychological and behavioral segments — alongside the demographic characteristics of the population.
And then I can run a poll using the psychological segments, and I can run the poll using just raw data. And I can run it with both. And I can compare and contrast how powerful [the psychological data is for predicting] how people are going to vote.
Then I can come back and I can look at 2016 results. I can look at the referendum results, and see if there’s any trend. There might be nothing, I could do all this work and there’s nothing there, right? But I just, I just, I just I’ve felt for about a year and a half now that I'm missing something by just using data.
Brian Resnick
So let’s say the polling is showing one candidate is winning. But then, you find some very strong emotional resonance for the other candidate. Would that strong emotional resonance make you have less faith in the poll? Is that what you’re getting at?
Ian Warren
Yes, that’s what I'm working toward.
Brian Resnick
This reminds me of the debate in the media about the value of on-the-ground reporting. The traditional model was sending a reporter out to middle America to take the pulse of a community. Some would say that’s the best way to find out what the country is feeling. Other people would say it’s too costly and you could just write yourself in anecdotal traps.
And it sounds like what you’d like to do is take that emotional pulse that reporters would try to get and make it methodically rigorous.
Ian Warren
Yes, absolutely, yes. A key thing for me, like a bit of a break-free moment for me because I’ve been in my office for 10 to 15 years just doing models behind my computer screen.
So there’s this big disconnect between what data analysts and political parties believe cuts through and what actually does cut through.
