Radiology AI’s CAD Déjà Vu: Lessons for Investors

The hype around diagnostic AI is deafening, but for anyone who’s been around long enough, it feels a lot like déjà vu. The way modern radiology AI is developing, and especially how it’s failing in the early going, is almost a mirror image of the long, painful journey of Computer-Aided Detection (CAD) in mammography. It’s a story packed with lessons about what happens when you trust lab performance metrics over actual clinical utility.

The Unfulfilled Promise of Early Mammography CAD

When CAD systems for mammography first got FDA 510(k) clearance back in the 1990s, companies like Hologic and iCAD pitched them as a revolution in cancer detection. The idea sold itself: an automated second reader that could flag tiny abnormalities a human might miss, boosting diagnostic sensitivity. Backed by new Medicare reimbursement codes that paid for its use, this promise drove adoption through the roof. The problem was, the expected jump in better patient outcomes never really happened. Instead, a wave of retrospective analyses and long-term observational studies, including some funded by the National Cancer Institute (NCI), showed a much darker picture. Sure, CAD was finding more “stuff,” but it was also causing a huge spike in false-positive rates Journal of the National Cancer Institute study on CAD effectiveness. That meant more benign findings got flagged, leading to a cascade of unnecessary callbacks, extra imaging, and biopsies that loaded up patients with anxiety and the healthcare system with costs. Radiologists weren’t being helped. They were being forced to wade through a sea of CAD alerts, most of which were clinically meaningless. This led to massive alert fatigue, and the net result was often just more time spent per read with no real improvement in finding the cancers that mattered.

The Data Speaks: CAD’s Impact on Diagnostic Accuracy and Workflow

The critical lesson from CAD’s history is right there in the numbers. Study after study showed that while CAD might give a tiny bump to sensitivity in some cases, it came with a completely disproportionate increase in false positives. That trade-off basically destroyed its value. For example, research in the Journal of the American College of Radiology showed CAD’s actual benefit on true positive rates was minimal, while its contribution to false positives was huge American College of Radiology findings on CAD. This put radiologists, who were already struggling to keep up with volume, in the position of evaluating even more data points, most of which were just noise. The workflow was just as bad. Instead of making the diagnostic process simpler, CAD added a whole new layer of cognitive work. Radiologists had to burn precious minutes deciding whether each CAD mark was worth chasing or could be dismissed. This extra reading time, a make-or-break metric in any busy radiology practice, completely wiped out any theoretical efficiency gains. Even under the tight oversight of the Mammography Quality Standards Act (MQSA), CAD’s performance in the wild was nothing like what was promised in the lab. The dream of an AI partner that would augment human skill soured into a system that created more questions than answers, breeding skepticism among clinicians and forcing a total rethink of its value.

Identifying Modern Radiology AI Failure Modes Through a Historical Lens

For any healthcare VC, private equity investor, or radiology practice leader sizing up a diagnostic imaging company, the CAD saga isn’t just a history lesson. It’s a blueprint for spotting the exact same pitfalls in today’s AI startups. The failure modes are disturbingly similar:

• Overemphasis on Sensitivity in Isolation: A lot of new AI tools come out swinging with impressive sensitivity stats pulled from curated datasets. But just like with CAD, if those gains come with a flood of false positives or if the algorithm’s performance degrades when it sees a diverse, real-world patient population (a process called algorithmic drift), its clinical usefulness plummets. Investors have to look at specificity and positive predictive value just as hard as they look at sensitivity.
• Alert Fatigue and Workflow Disruption: An AI that generates a constant stream of alerts, even if some are legitimate, will create so much noise that clinicians either start ignoring everything or waste all their time triaging garbage. This is exactly what happened with CAD, where radiologists just stopped trusting the marks because the false-positive rate was so high. A good tool has to integrate into the existing workflow and actually reduce the cognitive burden on the doctor.
• Lack of Prospective, Real-World Comparative Data: Many AI products get their 510(k) clearance based on retrospective studies or by comparing their results to old data. CAD proved that this kind of data is a terrible predictor of how a tool will perform in a busy, real-world clinic. It was the NCI’s long-term observational studies that gave us the real picture, uncovering problems that the initial pre-market tests completely missed. So the question is, where’s the real-world proof? Investors should be demanding evidence from prospective, head-to-head trials that show better patient outcomes or real workflow savings, not just better scores on a test set.
• Unclear Reimbursement Pathways and Value Proposition: A CPT code might exist, but the AI’s value has to show up as a real benefit for the people paying the bills. If an AI’s main effect is to drive up downstream costs with things like unnecessary follow-up scans and biopsies, without a clear win in diagnostic accuracy or patient management, its reimbursement is going to be on shaky ground. That’s precisely the re-evaluation that happened with CAD once its limited clinical benefit became obvious.
  

  The Investor’s Mandate: Demanding Strong Evidence

  The lesson from mammography CAD is simple: sensitivity claims tested in a lab, while a necessary starting point, tell you almost nothing about real-world clinical utility or commercial viability. Investors looking at radiology AI today need to be far more rigorous in their due diligence. It means you have to look past the slick AUC curves and demand evidence that directly answers the hard questions that the CAD experience raised. Companies like Hologic and iCAD spent decades learning these lessons and tweaking their algorithms. Today’s AI-native companies have the benefit of more advanced deep learning and bigger datasets, but the basic principles of clinical validation haven’t changed one bit. Investors ought to prioritize startups that can show:

• Balanced Performance Metrics: High sensitivity that’s backed up by strong specificity and a low false-positive rate when tested across different kinds of patient populations.
• Workflow Integration and Efficiency: Proof that the solution actually cuts down radiologist reading time or improves their confidence without creating new headaches or slowdowns.
• Prospective Clinical Validation: Hard evidence from properly designed prospective studies that compare the AI-assisted workflow to the current standard of care, preferably with patient outcome data.
• Clear Economic Value: A return on investment that a hospital CFO can understand, whether it’s through better patient outcomes, lower overall costs, or more efficient operations. The history of CAD isn’t a condemnation of AI. It’s a guide. By understanding its failures, investors can get much better at figuring out which of today’s AI tools are actually set up for long-term clinical impact and commercial success, and which ones are just repeating the mistakes of the past. This analysis is based on a synthesis of peer-reviewed retrospective studies on Computer-Aided Detection in mammography, drawing upon historical data from organizations such as the National Cancer Institute and the American College of Radiology, and regulatory filings with the FDA. FDA 510(k) historical database for CAD devices.