:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/47554101/GettyImages-490567928.0.jpg)
Elizabeth Holmes, founder of the embattled biotech startup Theranos, managed to assemble an influential board, attract a $9 billion valuation, and win accolades from the likes of President Obama and Time magazine — with promises that she could disrupt the multibillion-dollar US blood testing industry with novel pinprick tests.
Recently, however, questions about the reliability and accuracy of Theranos's lab testing arose with an investigation in the Wall Street Journal. Reporter John Carreyrou found that many of the company's revolutionary claims — that it could deliver high-quality test results from just a tiny drop of blood faster and more cheaply than incumbent companies — were overblown and that, behind the scenes, the company was basically operating like a traditional lab. (You can read about the whole controversy here.)
Long before reports about Theranos's dubious practices hit the mainstream media, scientists were calling the company's claims into question in these two particularly prescient papers.
Theranos never published data proving their technology worked
Theranos was skillful at attracting favorable coverage from mainstream media outlets. But the company never backed up those claims in peer-reviewed journals, where they would have received more rigorous analysis from others in the medical community. When asked about the technology, Holmes and Theranos's PR team deflected questions by citing intellectual property concerns and insinuating that any complaints were being planted by rival testing companies such as Quest and Laboratory Corporation.
Other non-healthcare startups may not need to prove their technology by publishing their data, but that's not true in the highly regulated medical space. According to researchers who study diagnostic technology, Theranos's approach was highly unusual. In May, a Toronto pathologist and lab medicine researcher, Eleftherios Diamandis, flagged this issue in the journal Clinical Chemistry and Laboratory Medicine:
The quality of the results are not known since the Theranos system has not been independently evaluated, nor do any published results exist to compare with conventional technologies. New diagnostic tests must be evaluated for their accuracy, precision, specificity and long-term robustness. Trueness and precision (accuracy) need to be maintained over months or years, and monitored by external quality assurance programs, so that patient’s data can be directly com- pared over long periods of time. Without independent validation, Theranos technology’s quality and robustness will remain in question.
He concluded by stating that Theranos's "claims of superiority over current systems and practices are speculative, at best."
In another paper, published in February in the journal JAMA, Stanford researcher John Ioannidis also pointed out that this secretive approach was odd and made it difficult to trust Theranos:
Stealth research creates total ambiguity about what evidence can be trusted in a mix of possibly brilliant ideas, aggressive corporate announcements, and mass media hype. The unquestionable success of computer science, engineering, and social media technologies has created reasonable hope that these technologies can also improve health in ways that the biomedical and life sciences have failed to do until now. But then how can the validity of the claims made be assessed, if the evidence is not within reach of other scientists to evaluate and scrutinize?
In the absence of more scientific transparency, "investors, physicians, patients, and healthy people will not be able to judge whether some proposed innovation is worth $9 billion, $900 billion, or just $9 — let alone if the innovation will improve the health and well-being of individuals."
The skeptical tone of these papers is vastly different from the tenor of positive coverage in most of the mainstream media until a few weeks ago. In a best-case scenario, Theranos will publish its data, as they recently promised to do, and concerns about their accuracy and reliability will be proven unfounded. In the worst case, this will be another example where the public fell prey to an over-hyped health promise supported by no good evidence.
