clock menu more-arrow no yes mobile

Filed under:

AI can now outperform doctors at detecting breast cancer. Here’s why it won’t replace them.

Google has developed an impressive new algorithm for analyzing X-rays.

A woman being given a mammogram. Getty Images
Sigal Samuel is a senior reporter for Vox’s Future Perfect and co-host of the Future Perfect podcast. She writes primarily about the future of consciousness, tracking advances in artificial intelligence and neuroscience and their staggering ethical implications. Before joining Vox, Sigal was the religion editor at the Atlantic.

Breast cancer affects way too many of us. In the US, one in eight women will develop it in their lifetimes. But encouraging new research shows that artificial intelligence can help with early detection.

An AI system developed by Google Health, Google-owned DeepMind, and several medical centers is so good at detecting breast cancer that it can outperform actual doctors, according to a paper published this week in the journal Nature. The AI analyzes mammograms — the X-rays commonly used to check for breast cancer — to determine whether the disease is present.

Researchers found that the AI system reduced false positives by 5.7 percent for US women — a significant improvement, when you consider how distressing it would be to be told you have cancer when you actually do not. It also reduced false negatives by 9.4 percent, meaning it caught instances of cancer that would’ve otherwise gone undetected.

And it did this by “looking” at mammograms alone, without access to any of the other health data that human doctors have on their patients.

This does not mean AI will soon replace radiologists — that’s a common but false narrative. While AI systems catch things that doctors miss, doctors also catch things that AI systems miss. Their abilities are complementary, best used together.

To build the AI system, researchers got anonymized mammograms from some 76,000 women in the UK and 15,000 women in the US. They used that data to train the system. Then they tested it on the X-rays of a different group — 25,000 UK women and 3,000 US women — analyzing how often the AI was right about whether the woman actually ended up having cancer, as determined by biopsies and follow-ups.

In both the UK and the US, the AI outperformed a single radiologist.

In the UK, the standard of care is to have two radiologists read the X-rays, which can be tricky to analyze. The AI didn’t do better than two radiologists combined, but it didn’t do worse than them, either — and it reduced the workload of the second reader by 88 percent.

Since there’s a shortage of radiologists in the UK (among other countries), that kind of time-saving could mean more patients get seen quicker and cancer gets detected and treated earlier in the patients who need it most.

It’s the latest example of how AI is changing medicine in ways that can improve diagnostics and triage. Google researchers have already developed an AI system that is impressively accurate at detecting some types of eye disease. AI can also recommend the correct treatment approach for more than 50 eye diseases. Plus, algorithms are being used to diagnose polyps on the colon during colonoscopies, to detect lung cancer, to understand how proteins fold so we can develop new drugs, and more. It may not be long before AI yields tangible benefits for your health.

This is promising research, but let’s not get ahead of the science

It’s exciting to watch AI make progress in the medical realm, especially when it comes to a disease like breast cancer, which kills more than half a million people around the world each year. But AI is no panacea, and it’s important to understand the limitations and risks that come with it.

For starters, there’s a big difference between getting an AI system to perform well retrospectively — where the patient’s final diagnosis is already known, as in Google’s breast cancer study — and getting it to make accurate guesses about current patients whose diagnoses are still unknown.

“Prospective studies are the only way you find out how these things perform in the real world,” Christopher Kelly, a co-author on the study, told Wired. Google Health director Dominic King added that prospective studies are coming. “That’s a different program of research that we’re now excited to be exploring.”

Just because an algorithm appears to work great in a computer simulation doesn’t mean it’ll work as intended in all doctors’ offices. One thing to watch out for is how the availability of an AI system affects the attention and confidence of a human doctor. If doctors grow accustomed to relying on the AI because it’s usually right, they might miss an instance of cancer that they normally would’ve spotted.

Another question we have to ask about any AI system is whether it’s generalizable: If you train it on data from one population in one country, will it work as effectively when used on another population in another country?

To evaluate this, the Google researchers trained their AI using only the UK mammograms and applied it to the US. Even without exposure to the US training data, the AI system outperformed radiologists — an encouraging result. Knowing that AI systems sometimes perform less well on minority groups, the researchers also checked for algorithmic bias, but found no evidence of it. Unfortunately, the racial background of the women in the study was not specified.

Also unfortunate: The source code for the algorithm was not made publicly available. Being transparent rather than releasing black-box algorithms would be preferable for scientists as well as doctors and their patients. Transparency means other scientists can build on the work and doctors can rest assured that they understand how and why an AI arrived at its decision, allowing them to reassure patients in turn. But too often, AI companies treat their data as proprietary.

Finally, questions about privacy should always be at the forefront of our minds when a company — especially a tech giant like Google, which already has tons of data on us and which is expanding into health care — begins amassing our medical records. In 2019, a whistleblower alleged that Google Health had been gaining access to millions of Americans’ medical data without it being anonymized. It’s crucial that such sensitive data be de-identified. The researchers behind the breast cancer study took care to note that theirs was.

Sign up for the Future Perfect newsletter. Twice a week, you’ll get a roundup of ideas and solutions for tackling our biggest challenges: improving public health, decreasing human and animal suffering, easing catastrophic risks, and — to put it simply — getting better at doing good.

Sign up for the newsletter Today, Explained

Understand the world with a daily explainer plus the most compelling stories of the day.