Flipping Google's image recognition AI produces some amazingly trippy images

Image recognition technology is pretty powerful stuff. It lets Facebook auto-tag photos of your friend, Google Images show you photos that look similar to a photo you already have, and self-driving cars recognize pedestrians before hitting them. It also, crucially, lets Google generate totally whacked-out, trippy masterpieces like this:

Mike Tyka / Google

Alexander Mordvintsev, Christopher Olah, and Mike Tyka, three software engineers at Google, explain at the company's research blog that the above image comes from flipping around an "artificial neural network" — a kind of artificial intelligence system that mirrors the structure of biological nervous systems — that does image recognition. Rather than taking in an image and trying to see what objects are contained within it, the flipped neural net takes an image and tries to "see" it in such a way that an object it already knows about emerges. The photo above was the result of feeding random noise to a neural net trained to recognize "places."

They can also find more specific things. This is what happened when they asked the neural net to find a banana in a bunch of meaningless pixels:

Mike Tyka / Google

Pretty cool, huh? Here are a few more photos generated using the same place-recognizing net as the top image:

Mike Tyka / Google

Mike Tyka / Google

Mike Tyka / Google

Mike Tyka / Google

You can also send representational images through nets that have been trained to recognize different sorts of images, thus creating a new image that combines them both. For example, here's an image of a knight as interpreted by a neural net trained to recognize animals. The end result is a knight that's been morphed into a chimera of several dogs:

Mike Tyka / Google

"Flipping" image recognition neural nets is an important for understanding how, exactly, they're recognizing objects. If you want a neural net to recognize dumbbells, for example, you might send it a bunch of images of people doing arm curls with dumbbells. Ideally, the nets will just notice the weights. But sometimes, as in the case of one Google neural net, they pick up too much, and think that all dumbbells have to have muscly arms attached:

Google

By inverting this neural net, Google learned that there was a big flaw in how it was identifying dumbbells, which it could then fix to improve the net's recognition powers. As the Guardian's Alex Hern notes, in this example, it might give them reason to feed the net images of dumbbells sitting still on the ground, so that the net dissociates the concept of dumbbells from the arms of people holding them.

Here are a few more fun digital hallucinations that Google's neural nets have produced:

Mike Tyka / Google

Mike Tyka / Google

Left: Original photo by Zachi Evenor. Right: processed by Günther Noack, Software Engineer. Via

Google

Thanks to Hern at the Guardian for the pointer.

Will you support Vox’s explanatory journalism?

Most news outlets make their money through advertising or subscriptions. But when it comes to what we’re trying to do at Vox, there are a couple of big issues with relying on ads and subscriptions to keep the lights on:

First, advertising dollars go up and down with the economy. We often only know a few months out what our advertising revenue will be, which makes it hard to plan ahead.

Second, we’re not in the subscriptions business. Vox is here to help everyone understand the complex issues shaping the world — not just the people who can afford to pay for a subscription. We believe that’s an important part of building a more equal society. And we can’t do that if we have a paywall.

So even though advertising is still our biggest source of revenue, we also seek grants and reader support. (And no matter how our work is funded, we have strict guidelines on editorial independence.)

If you also believe that everyone deserves access to trusted high-quality information, will you make a gift to Vox today? Any amount helps.

One-Time Monthly Annual

$5/month

$10/month

$25/month

$50/month

Other

$

Yes, I'll give $5/month

Yes, I'll give $5/month

We accept credit card, Apple Pay, and Google Pay. You can also contribute via