How Twitter taught a robot to hate

It took 15 hours for Twitter to teach an artificially intelligent chatbot to be a racist, sexist monster.

If you're shocked, you probably don't spend much time on Twitter. Microsoft's programmers presumably do, though, and the shocking thing is that they didn't see this coming.

Microsoft created a chatbot named Tay that was designed to talk like a millennial and learn more authentic conversation by interacting with humans online. Tay was dubbed by her creators as an "AI fam from the internet that's got zero chill!" (Oh boy.)

Some of Tay's interactions might actually pass a millennial Turing test. She even uses emoji!

Unfortunately, millennials are also just about as racist and possibly even more sexist than their parents. It didn't take long before Tay was imitating those qualities too, and a cute 19-year-old girl transformed into a Gamergate-loving Hitler youth.

Twitter trolls started teaching Tay some horrible racial slurs and genocidal ideation.

The latter tweet, Business Insider notes, appears to be the result of a user asking Tay to repeat a phrase verbatim. Some, but not all, of Tay's offensive tweets did something like this.

Tay also started harassing Zoë Quinn, a game developer who once went into hiding due to the virulent misogynistic "Gamergate" threats she received online.

Tay also started hitting on random people in direct messages.

Quinn, and many other critics, pointed out that Microsoft's designers really should have anticipated these outcomes and programmed Tay with filters ahead of time.

Microsoft has deleted most of the offensive tweets, and told Business Insider that it's now making "adjustments" to the bot.

Microsoft's website for Tay featured this banner on Thursday morning:

Twitter itself has also come under heavy criticism for not doing enough to address harassment, especially of high-profile users who are women and people of color. Some leave the platform because the problem is so bad. Twitter has made some changes, but many users still charge that the company isn't making harassment enough of a priority. This may be partly because its staff isn't very diverse, and the problem may feel less urgent to white men whose lives aren't severely impacted by racialized, sexual harassment.

The same general problem may be at work here. The possibility of harassment is going to be more top of mind for women and people of color who experience it frequently, but they're also less likely to be well-represented on tech teams. Microsoft is no exception.

Or maybe Microsoft just somehow failed to take basic precautions that should be standard in the industry.

Microsoft said it created Tay to "experiment with and conduct research on conversational understanding." The engineers probably came away with a different understanding of conversation than they bargained for.

Contributions are a key part of the future of Vox

Readers rely on Vox for clear, nuanced coverage that not only illuminates the issues, but poses solutions, too. And we rely on help from our readers: Advertising and grants cover the majority of our costs, but we count on contributions to help us close the gaps in our budget. In fact, we’re looking to reach 95,000 individual contributions before the end of the year. Will you make the next contribution right now? Our average gift is just $20 — and it goes a long way in helping us keep our work free. Vox is here to help everyone understand what’s shaping the world — not just the people who can afford to pay for a subscription. We believe that’s an important part of building a more equal society. Join that mission by making a contribution today.

One-Time Monthly Annual

$5/month

$10/month

$25/month

$50/month

Other

$

Yes, I'll give $5/month

Yes, I'll give $5/month

We accept credit card, Apple Pay, and Google Pay. You can also contribute via