Book publishing’s fact-checking failure, as illustrated by the Sally Kohn controversy

CNN political commentator Sally Kohn published a new book, The Opposite of Hate, earlier this month. It’s a book devoted to the laudable cause of understanding and then fighting back against hatred throughout the world, especially in the form of bigotry and prejudice. So far so good, right?

Here’s the problem: One chapter features a fairly explosive quote from Call Your Girlfriend podcast host Aminatou Sow. Kohn quotes Sow as saying that if she is provoked by a Twitter troll, “I can fucking kill you and I’m justified in doing that.” Metaphorically speaking, Kohn adds.

Sow says that she never said those words. Moreover, she says that she never went on the record with Kohn. Kohn, for her part, maintains that Sow did say those words and that she did go on the record with them, but as a courtesy, she has removed the quote from the digital edition of her book and has committed to removing it from any subsequent printed editions.

The entire controversy is a deeply vexed situation that’s become a referendum on liberal racism and on the way that black women are used as props by progressives. (For more on its social implications, I’d recommend takes by Amanda Arnold at the Cut and Amy Thomson at Mother Jones.) But the controversy has also brought up one of the knottier and more confusing questions that periodically rears its head in book publishing.

Sow says that the misquote would have been caught in fact-checking if anyone had contacted her to verify it. Kohn says that she fact-checked the quote by checking it against her notes and that this method is the journalistic standard.

So how does fact-checking work in book publishing? And is the way that it works effective?

The standard rule in book publishing is that accuracy is the author’s problem

In general, fact-checking is not a standard part of the workflow in book publishing, even in nonfiction book publishing. What usually happens is this: Authors submit their manuscripts, the manuscripts go to editors who help to refine them and shape them, and from there the book goes into production and copy editing.

The copy editor will look for grammatical errors, and sometimes the publisher’s lawyer will check the book to make sure there’s nothing libelous in there, but fact-checking is not part of the standard publisher’s process.

There have been exceptions to this rule. In 2015, Vulture reported that Penguin Random House imprint Tim Duggan Books “has been quietly promoting in-house fact-checking as a special feature of his new shop,” and confirmed that at least one of the books Tim Duggan published had been fact-checked at the publisher’s expense. But the Tim Duggan fact-checking project was an exception, not the norm, and it’s not clear that it survived into 2018. (The imprint did not respond to requests for comment.)

So how do publishers generally handle it if factual errors creep into a book? Basically, the same way they handle plagiarism: They make it the author’s problem.

One of the standard parts of any book contract is the warranty and indemnity clause. By signing on to that clause, an author is guaranteeing that their book is their own, original project, not plagiarism, that it doesn’t infringe on anyone else’s rights, and — if the book is nonfiction — that its facts are accurate. And if it turns out that any of these claims are untrue, the liability is all on the author. They’re the ones who pay up if someone decides to sue.

So the facts are all up to the author. And different authors handle that liability differently. Some might want to hire a freelance fact-checker, but that can get expensive: Vulture cites flat prices of between $5,000 and $25,000, and the Editorial Freelancers Association quotes a rate of about $30 to $40 per hour. The money for fact-checker fees would have to come from the author. And since most nonfiction book authors aren’t exactly rolling in spare cash, it’s a tempting corner to cut. Many authors decide to just fact-check themselves or to skip that step entirely.

Either way, we’re left with an industry in which a lot of nonfiction books don’t get looked over by a professional fact-checker.

“While I can’t speak for every publisher or author, my experience with book-length material is that fact-checking is the exception, not the rule,” says Rob Liguori, a New York-based freelance fact-checker with nearly a decade of experience.

A fact-checker wouldn’t have fixed every issue in the Kohn/Sow mess, but it might have helped a little

According to Sow, she had a single private conversation with Kohn in the back of a car last summer, after drinks. At no point, she says, did she agree to go on the record for Kohn, and at no point did she see Kohn record or take notes on their conversation.

The fact that no one ever contacted Sow to verify her quote after the fact is a major problem, but it’s not the main issue. It’s part of what Sow has framed as the larger problem: According to Sow, Kohn violated basic journalistic principles. (Sow did not respond to a request for comment.)

For her part, Kohn maintains that she did ask for Sow’s permission to put her on the record and that she did take notes on their conversation on her iPhone as they spoke. (She’s put a copy of her notes on Twitter.) Moreover, Kohn says, she did fact-check the quote: she checked it against her notes.

“That is the standard of journalism. That’s the standard of news reporting. That’s how it’s done,” Kohn said in an interview on the Twitter BuzzFeed morning show AM to DM. “Every Times story you read, every Washington Post story you read, they’re not going back and checking with people who went on record and saying, ‘Are you okay with your quotes?’ … Let’s be clear: I’m not inventing a new standard of book-writing here. It’s false to say that the book wasn’t fact-checked. And it’s false to say that the quotes weren’t checked. They were checked against the notes.”

Over the phone, Kohn reiterated her statements. “I have tried to be direct, forthcoming and honest, including about what I wish I had done differently,” she told Vox, saying that she felt that as a white woman she should have been more careful about quoting such an explosive line from a woman of color and that she feels that the critiques of the racial politics of the quote are valid.

But, she said, she continues to maintain that she asked for and received consent to put Sow on the record and that she visibly took notes on the conversation as they spoke. And, she added, “It is true that there is not an industry standard of confirming quotes.”

It’s certainly the case that there are multiple standards of quote confirmation and fact-checking in journalism. As writer Ann Friedman pointed out on Twitter, in a thread that’s critical of Kohn, “Standards for how (and how rigorously) to check facts vary quite a bit depending on what corner of the industry you’re in, what outlet you work for, how tight your deadline is, and even, sometimes, who’s editing you.”

Fact-checking quotes for a newspaper story that has to be published in a matter of hours is a different process than fact-checking a long reported article that has a lot of time and resources going into it, which is part of why magazine fact-checking is more rigorous than newspaper and news website fact-checking. (Disclosure: Vox does not have a dedicated fact-checker on staff, though our copy editors do fact-checking for large, reported articles, and we occasionally hire third-party fact checkers for especially sensitive articles.) And fact-checking a quote transcribed during casual conversation with an acquaintance is a different process from fact-checking a quote gathered in a formal interview.

Quote approval, in which a source gets to read their quotes verbatim and then veto what shows up in print and what does not, is a widely disliked practice that reputable outlets use only in special circumstances. But checking the substance of a quote with a source for accuracy is a different matter.

“I endeavor to verify every quotation in a book, and there is a hierarchy of what constitutes good backup for quotes,” freelance fact-checker Liguori told me. “Most effective would be a discussion with the source about the substance of the quote — though I do not read back quotations to sources verbatim because of concerns about ‘source remorse’ — followed by checking the quotation against an audio recording or a professionally-prepared transcript. Reliance on the author’s contemporaneous, often piecemeal notes is the least reliable way to check a quotation. (Dear writers: please tape your interviews!)”

So while Kohn’s book was fact-checked to an extent, it was not fact-checked with the rigor that an outside party might have brought to the process. And book publishing as an industry is set up to disincentivize bringing an independent fact-checker into the process.

All of which is to say that if Kohn’s publisher had made fact-checking a standard part of its workflow, the way that copy editing is, rather than making it the author’s problem, someone might have contacted Sow about her quote before the book went to press, and Kohn and her publisher might have avoided a public controversy.

A fact-checker would not have solved the central dispute here, which is about whether a journalist used ethical and accurate practices in getting a quote from a source on a sensitive subject. And using a fact-checker is not a magic solution that puts a book beyond all possibility of reproach.

But a fact-checker can act as an effective safety net, a support system that helps make a book the best and most accurate version of itself. And it’s a support system that, for the most part, the book publishing industry does not have.

That’s not a great situation for book authors, their sources, or their readers to be in.

Correction: An earlier version of this piece has Sow saying her conversation with Kohn took place years ago. Sow actually places it last summer.

Help us reach our year-end goal

In a recent reader survey, we were delighted to hear that people value Vox because we help them educate themselves and their families, spark their curiosity, explain the moment, and make our work approachable.

We believe that everyone deserves access to clear, factual information that helps them educate themselves on the issues of the day and the things that pique their curiosity. That's why we're committed to keeping Vox free.

Reader gifts enable us to offer free articles, videos, and podcasts to everyone, while relying less on advertising. In service of that mission, we're aiming to add 3,000 new gifts from readers by December 31.Will you help us reach our goal, and continue to keep Vox free, by making a gift today?

One-Time Monthly Annual

$95/year

$120/year

$250/year

Other

$

Yes, I'll give $120/year

Yes, I'll give $120/year

We accept credit card, Apple Pay, and Google Pay. You can also contribute via