Why so many polls get American attitudes about abortion wrong

Is the public really divided on abortion? It’s surprisingly hard to know.

I’ve studied public opinion toward abortion on and off for about a decade as a researcher. Recently I was conducting focus groups where we specifically recruited people who had told us abortion should only be legal in cases of rape or abuse, or if the mother's health is at risk. But once the conversation started, the issue quickly got more complex. Participants would say things like, "The government should stay out of the decision. It is up to the woman."

Read more: What Americans think of abortion isn't black and white

As a pollster, I find this perplexing: these are people who say abortion should only be legal in rare cases, but also want the government out of the decision?

That focus group reminded me of a survey I conducted a few years ago. Some of the respondents who said abortion should only be legal in those rare cases also opposed some restrictions on abortion.

These are just a few of the contradictions I’ve seen, having surveyed thousands of people across the country on the topic. The biggest lesson I’ve learned is this: the current polling fails at accurately measuring opinion on this complex issue.


I got an opportunity to dig deeper into the data in a recent poll we conducted for Vox. What I found is the standard measures used to report the public’s views on abortion — polling questions used by Pew Research Center, Gallup, Washington Post-ABC News, our firm, and others — are, at best, likely flawed — they don’t capture how people really think.

The standard measures ask respondents about when or in what cases abortion should be legal. The question wording and response categories vary across pollsters. But when collapsed into two categories — legal and illegal — you tend to get a divided public.

Here’s how Pew asks the question.

"Do you think abortion should be legal in all cases, legal in most cases, illegal in most cases, or illegal in all cases?"

In 2013, Pew reported 54 percent of the public said abortion should be legal in all (20 percent) or most (34 percent) cases and 39 percent said abortion should be illegal in all (15 percent) or most (24 percent) cases. In 2014, Gallup asked similar questions and also found the public fairly split (although in the reverse direction).

Our Vox poll shows a similar divide: close to half (46 percent) said abortion should be legal in "almost all" or "most" cases. Fifty percent said abortion should be legal "only in cases of rape, abuse, or if the woman’s health is at risk" (34 percent) or "never legal" (16 percent).

But in our poll, we dug further: we looked specifically at the 34 percent of respondents who said abortion should only be legal in cases of rape, abuse, and health risk. And we asked them a follow-up question about their view of Roe v. Wade. This was the exact question: "In 1973 the Roe v. Wade decision established a woman’s constitutional right to have an abortion. Would you like to see the Supreme Court overturn its Roe v. Wade decision, or not?"

Pro-choice versus pro-life: it's not that simple

Gender
Age
Race/ethnicity
Political party
Overturn Roe
Gender
Abortion demographics toggle gender
Age
Abortion demographics toggle age
Race/ethnicity
Abortion demographics toggle race ethnicity
Political party
Abortion demographics toggle political party
Overturn Roe
Abortion demographics toggle overturn Roe

Half (53 percent) said they do not want to overturn Roe v. Wade. Keep in mind: these are people who say they think abortion should only be legal in rare circumstances. Wouldn’t you expect a vast majority of them to disagree with the Roe decision?

But these seeming inconsistencies show up again and again. We asked about the recent wave of 205 abortion restrictions passed across the country. We had a hunch the public did not know of the trend, so we presented information about the types of restrictions and then asked if they felt the trend is going in the right or wrong direction.

Forty-five percent of those who said abortion should be legal only in rare cases also said recent restrictions are going in the wrong direction. One in five (21 percent) who said they want abortion illegal in all cases said restrictions are going in the wrong direction.

We also asked an open-ended question after presenting information about the types of restrictions: "What is one word that describes how you feel about these restrictions?"

Among some respondents who said abortion should only be legal in rare cases, we got responses like: "needed," "overdue," "thrilled," "encouraging," and "agree." But we also got: "bullshit," "ridiculous," "scared," "terrible," "tragic," "bad," "ashamed of our country," "offensive," "crazy," "disturbing," and "horrified."

I had my colleague go back three times to check this data because it was so surprising: it revealed that when you dig beneath the standard measure, you find strong emotions attached to the opposite view you would expect.

It’s not just contradictions between the standard measure and other questions about laws.

When it comes to "real life" views on the issue — how people actually experience abortion — the numbers get even more intriguing. Among people who said abortion should only be legal in rare cases, 71 percent said they would give support to a close friend or family member who had an abortion, 69 percent said they want the experience of having an abortion to be nonjudgmental, 66 percent said they want the experience to be supportive, 64 percent want the experience to be affordable, and 59 percent want the experience to be without added burdens.

So what is going on? When some respondents say abortion should be legal only in rare cases, are they thinking of an ideal but unrealistic world? Are they answering the question in personal terms — cases in which they personally would consider an abortion? But in reality, as in my focus groups, they want each person to decide rather than the government?

Regardless of what respondents are thinking, we, as pollsters, need to rethink how we measure public opinion on this issue. At minimum, we need to do better at avoiding measurement error.


Now is an especially important time to try to get it right.

States are briskly passing laws that are closing clinics and limiting access. In the past three months, more than 200 anti-abortion bills have been proposed in state legislatures across the country. Republicans in Congress are attaching anti-abortion legislation wherever they can — very recently to a bill about doctors’ reimbursements for Medicare patients.

We need to ask questions about how the public views abortion policy — but do so in a more real and accurate way. We shouldn’t, for example, simply ask "Do you support or oppose recent restrictions to abortion?" when we know most people aren’t aware of any trend or what the restrictions might be.

Maybe one place to start is moving beyond legality and into reality. Although we don’t talk about it much, one in three women in this country will have an abortion in her lifetime. Maybe pollsters should learn more about what the public knows about access and what they want in place for a woman who has decided to have an abortion. After all, it has been legal "in most cases" for 42 years.

PerryUndem Research/Communication conducted the survey among n = 1,067 adults 18 and older nationwide, March 4 through 12, 2015. The survey was administered among a nationally representative sample of adults, using GfK's Knowledge Panel. The margin of error is +/– 4.1 percentage points. Some results do not add to 100 percentage points as a result of rounding. Topline results are available here.


Editor: Susannah Locke, Sarah Kliff
Designer: Tyson Whiting
Developer: Yuri Victor
Charts: Anand Katakam, Tyson Whiting