clock menu more-arrow no yes mobile

Filed under:

An AI system competed against a human debate champion. Here’s what happened.

IBM’s “Miss Debater” lectured a Silicon Valley audience on morality.

Moderator John Donvan, IBM researcher Noam Slonin, Project Debater manager Ranit Aharonov and debate champion Harish Natarajan at IBM’s Think 2019 conference.
MediaNews Group/The Mercury News via Getty Images
Sigal Samuel is a senior reporter for Vox’s Future Perfect and co-host of the Future Perfect podcast. She writes primarily about the future of consciousness, tracking advances in artificial intelligence and neuroscience and their staggering ethical implications. Before joining Vox, Sigal was the religion editor at the Atlantic.

Remember your high school debate club? What happened in downtown San Francisco on Monday night was emphatically not like that.

In a highly unusual public debate attended by hundreds of eager onlookers, IBM pitted a female-voiced AI system, formally known as Project Debater but lovingly nicknamed Miss Debater, against Harish Natarajan, a debate champion who’s racked up more wins than any other person in the world. The goal was to see if an AI can beat a human debate pro.

Well, it turned out it can’t. At least not yet. But that’s okay: The most interesting applications of this debating technology, which has only been in development at IBM for six years, are yet to come.

The topic that Natarajan and Miss Debater argued over was “Should we subsidize preschools?” They each had 15 minutes to prepare. For Natarajan, that manifested as lots of scribbling on a pad of paper. As for the AI, the only sign of “thinking” was three spinning blue circles on the tall black box that is Miss Debater.

But inside the IBM machine, an invisible frenzy of activity was taking place: The AI was scanning 10 billion sentences (literally) in the hundreds of millions of documents to which it had access. After digging through all that information, it churned out a four-minute opening argument, a rebuttal, and a summary.

Miss Debater made a fairly sophisticated case that preschool should be subsidized. It argued that good preschools can help kids coming from underprivileged backgrounds break the cycle of poverty. Strikingly, it also made moral prescriptions, saying things like, “Giving opportunities to the less fortunate should be a moral obligation for any human being.” It wasn’t enough — Natarajan won.

Of course, the AI wasn’t inventing ethical positions of its own and trying to push them on us humans. It was just pulling from statements humans have already made and strategically spitting those back at us. Despite all the hand-wringing out there about the prospect of AI systems becoming our evil overlords, they’re not megalomaniacs, nor are they altruists — they’re just reflections of us in silicon, augmented with some pretty amazing computational power.

Why did the AI lose — and is that a problem?

One of the clever rhetorical tricks Natarajan used was to close the perceived gap between his opponent’s position and his. “First, I’d like to focus on what we agree on. Poverty is terrible,” he said. Then he argued that subsidies are “little more than a politically motivated giveaway to members of the middle class” and that they won’t really be an equalizer because they don’t benefit the most underprivileged.

When both of the debaters concluded their arguments, each audience member was asked to vote on which one they’d found most convincing. The human took the win.

The fact that Miss Debater lost the match-up may have been due in part to the lack of emotion in its delivery. It did display a sense of humor at points — for example, it kicked off the evening by telling Natarajan, “I have heard you hold the world record in debate competition wins against humans, but I suspect you have never debated a machine. Welcome to the future.” The audience laughed on cue.

But if you listen to the full video of the debate, you can hear that Miss Debater’s tone is pretty monotonous. That could make it harder to connect with listeners on a gut level and sway their feelings.

As Carmine Gallo pointed out in a write-up of the event for Forbes, Aristotle — who literally wrote the book on what makes for good rhetoric — insisted you need ethos (character), logos (reason), and pathos (feeling) to be a successful debater. If Aristotle could’ve seen Miss Debater in action, he probably would have said it was lacking in the pathos department.

Still, IBM’s achievement is impressive. Although we’ve already seen AI best human beings at games — chess, Go, and recently even StarCraft — those have clear, finite sets of preordained rules. Raw computational power works great in such cases because it lends itself to making choices among black-and-white options. But natural human discourse doesn’t work that way. As you may recall from high school debate club, it calls for deep nuance.

When you think about things that way, the fact that IBM developed an AI that could even approximate the world-record-holder’s level of skill starts to look more like success than failure.

Miss Debater is going into retirement for now as IBM moves on to developing the technology in more “commercially viable directions,” according to Ranit Aharonov, IBM’s manager for the AI system. For example, Aharonov said, this technology could help journalistic outlets or even governments lead people into nuanced debates about controversial topics that might otherwise get a more superficial treatment.

Hopefully, Monday’s debate will be remembered as a win for human beings — not just because human abilities won out, but because it successfully test-drove a technology with the potential to complement them.