By Editorial Team 
The echoes of Theranos’ spectacular collapse still reverberate through healthcare, a stark reminder of the perils of hype over evidence. Yet, the investment community continues to pour capital into unproven ventures, creating a new graveyard of promising but ultimately unsubstantiated healthcare AI companies. This pattern, from the blood-testing firm’s audacious claims to the recent struggles of Olive AI, raises a critical question for investors and patient safety advocates alike: what lessons remain unlearned about the imperative for rigorous clinical validation in health technology?
The Theranos Playbook: Billions Without Peer Review
The story of Theranos is now a cautionary tale etched into the annals of Silicon Valley. Founded on the promise of revolutionary blood testing from a single drop, the company attracted nearly $900 million in funding, reaching a peak valuation of $10 billion. The allure was undeniable: faster, cheaper diagnostics accessible to all. Yet, as the narrative unfolded, it became painfully clear that the technology simply did not work as advertised. The core issue, from a clinical and scientific perspective, was a profound lack of peer-reviewed evidence. Despite its massive capital infusion, Theranos never published robust data in reputable medical journals demonstrating the accuracy or utility of its proprietary Edison device. This absence was a glaring red flag for experts like Eric Topol, who consistently champions evidence-based medicine and the critical role of peer review in validating new health technologies Eric Topol on evidence-based medicine. The company’s operations ultimately drew the attention of the Department of Justice (DOJ) and the FDA’s Center for Devices and Radiological Health (CDRH), leading to its downfall. The Theranos saga demonstrated that even vast sums of capital cannot substitute for scientific rigor and regulatory compliance when patient safety is at stake.
Olive AI: A Modern Echo of Unsubstantiated Claims
Fast forward to the 2020s, and we see a similar, albeit less dramatic, pattern with Olive AI. This company, focused on automating administrative tasks in healthcare using artificial intelligence, raised an astounding $902 million and achieved a valuation of $4 billion at its peak. Its pitch resonated with investors seeking efficiency gains in a notoriously complex industry. However, just like Theranos, Olive AI faced significant scrutiny regarding the actual efficacy and return on investment of its solutions. Despite the impressive fundraising, the company struggled to provide concrete, peer-reviewed evidence of the substantial cost savings or operational efficiencies it promised. Healthcare systems that adopted Olive AI often found that the promised benefits did not materialize, leading to widespread dissatisfaction and eventual contract terminations. Ultimately, Olive AI ceased operations and dissolved in 2023, with its assets sold to other companies like Waystar and Humata Health. As noted by critics, including prominent voices in health innovation, the company’s aggressive growth outpaced its ability to demonstrate tangible value through validated outcomes. The relationship between massive capital raised and the absence of peer-reviewed evidence is a recurring theme (DP12, DP13). Both companies, despite their differing technologies, shared this critical flaw: they secured enormous investment without the foundational clinical validation that is non-negotiable in healthcare.
The Clinical Imperative: What Responsible AI Does Differently
For investors, the cases of Theranos and Olive AI serve as stark reminders that the “move fast and break things” mantra has devastating consequences in healthcare. Patient safety advocates, meanwhile, see these incidents as validation of their calls for stringent oversight. The question then becomes: what constitutes responsible AI in healthcare, particularly for those evaluating investment opportunities or considering adoption? The answer lies in a commitment to evidence. Responsible AI in healthcare demands rigorous validation through well-designed clinical studies, published in peer-reviewed journals. This includes transparent reporting of sensitivity, specificity, positive and negative predictive values, and the impact on clinical workflows and patient outcomes. For instance, a diagnostic AI tool should undergo studies akin to those published in The New England Journal of Medicine or JAMA, demonstrating its performance against established gold standards. Similarly, an AI-powered administrative tool, while not directly diagnostic, must still show empirically verifiable improvements in efficiency or cost reduction, with data published in health economics journals or equivalent. As Raj Komotar, a neurosurgeon and researcher, has emphasized in various discussions around AI in medicine, the excitement around technological advancements must always be tempered by the scientific method. The burden of proof lies squarely with the developer to demonstrate safety and efficacy. Harlan Krumholz, a leading cardiologist and health outcomes researcher, has consistently advocated for robust evidence generation, stressing that new technologies, especially those touching patient care, must be held to the highest standards of clinical validation, not just market hype Harlan Krumholz on clinical evidence in health tech.
Regulatory Context: The FDA SaMD Framework and Oversight
The regulatory landscape, particularly in the United States, is evolving to address the unique challenges of AI in healthcare. The FDA’s Software as a Medical Device (SaMD) Framework is a crucial guideline, distinguishing software that performs a medical function from general-purpose IT. SaMD products, depending on their risk classification, require varying levels of regulatory scrutiny, including premarket submissions like 510(k) clearances or De Novo classifications. The FDA CDRH plays a pivotal role in reviewing these submissions, ensuring that AI tools meet safety and effectiveness standards before reaching patients. The FDA has further evolved its approach to AI/ML-enabled medical devices, publishing an “AI/ML SaMD Action Plan” in January 2021 and final guidance on “Predetermined Change Control Plans” in December 2024, updated in August 2025, to manage modifications throughout a product’s lifecycle. Notably, the FDA also withdrew its “Software as a Medical Device (SaMD): Clinical Evaluation” guidance in January 2026. However, the cases of Theranos and Olive AI highlight a gap: while the FDA regulates devices, the broader investment community often overlooks the necessity of clinical evidence before massive capital deployment. The DOJ’s involvement in the Theranos case underscores that severe misrepresentations of medical technology can have criminal consequences, emphasizing the high stakes involved. Investors evaluating healthcare AI companies should scrutinize not just the technology’s potential, but its current standing within the FDA SaMD Framework and its progress toward generating the type of clinical evidence required for regulatory clearance and, more importantly, clinical adoption. A robust Quality Management System (QMS) aligned with standards like ISO 13485 and adherence to principles like Good Machine Learning Practice (GMLP) are critical indicators of a company’s commitment to safety and efficacy, which should be thoroughly vetted during due diligence.
The Path Forward: Evidence as the Ultimate Moat
The combined $1.8 billion invested in Theranos and Olive AI, largely without the backing of peer-reviewed clinical evidence, represents a significant misallocation of capital and a stark warning. For investors, the lesson is clear: clinical validation, published in reputable journals and scrutinized by the scientific community, is not merely an academic exercise; it is the ultimate de-risking strategy and a foundational requirement for sustainable success in healthcare AI. For patient safety advocates, these failures reinforce the critical need for vigilance and a demand for transparency and evidence from all health technology innovators. The future of responsible AI in healthcare depends on a collective commitment to evidence-based innovation, ensuring that groundbreaking technology truly benefits patients, rather than just enriching speculative ventures. Peer-reviewed evidence requirements for medical AI
Frequently Asked Questions
A4: What is the primary lesson investors should take from the failures of Theranos and Olive AI regarding healthcare AI investments?
The primary lesson for investors is that in healthcare, the ‘move fast and break things’ approach is dangerous. Both companies secured massive investments without providing rigorous clinical validation or peer-reviewed evidence to substantiate their claims of efficacy or value, leading to their ultimate downfall.
A4: What specific type of evidence should investors look for to identify responsible healthcare AI companies?
Investors should look for healthcare AI companies that demonstrate commitment to evidence through rigorous validation via well-designed clinical studies. This includes transparent reporting of metrics like sensitivity and specificity, and publication in peer-reviewed medical or health economics journals, proving impact on patient outcomes or verifiable efficiencies.
A5: How did the lack of clinical validation in Theranos and Olive AI endanger patient safety or healthcare operations?
In Theranos, the lack of clinical validation meant their diagnostic technology did not work as advertised, posing a direct threat to patient safety through inaccurate test results. For Olive AI, the absence of validated outcomes led to dissatisfaction and contract terminations from healthcare systems, indicating a failure to deliver promised operational benefits and potentially disrupting healthcare operations.
A5: What role should patient safety advocates expect regulatory bodies like the FDA to play in preventing future ‘graveyards’ of unproven healthcare AI?
Patient safety advocates should expect regulatory bodies to enforce stringent oversight and demand robust evidence of safety and efficacy for healthcare AI technologies. The FDA’s Software as a Medical Device (SaMD) Framework is a crucial guideline for distinguishing and regulating software that performs a medical function, ensuring new technologies meet high standards.
