Stop the Bleed: Auditing AI Drift in Cardiac Care

AI in healthcare promises a lot, but that value disappears fast if you ignore algorithmic drift. Clinical environments are constantly changing, patient populations shift, new treatments come online, and even rock-solid AI models will eventually degrade, leading to missed diagnoses, bad drug interaction advice, or undertriaging patients in the ED. It’s a quiet problem that requires active governance, and the big health systems are finally baking this reality into their operations.

The Inevitability of Algorithmic Drift and its Clinical Impact

Algorithmic drift isn’t some academic theory. It’s happening right now in hospitals. Take a diagnostic algorithm trained on data from 2018-2020. By 2026, it’s almost guaranteed to struggle as patient demographics and disease patterns change, or even when the radiology department gets a new scanner with slightly different imaging protocols. Suddenly you’re seeing more false positives, which means unnecessary and expensive follow-ups, or worse, more false negatives, which means a delayed or missed diagnosis. The data backs this up: a study from Harvard, MIT, The University of Monterrey, and Cambridge found that a staggering 91% of machine learning models degrade over time. This decay gets worse with changes in input data, like new imaging tech or different acquisition protocols. For example, one study saw an 18.7% drop in mean average precision when a model trained on one hardware system was tested on another, and a 22.3% degradation when applied across different demographic subgroups. The impact on patient safety and hospital resources is huge. An AI built to spot early sepsis might grow less sensitive, delaying life-saving interventions and producing poorer outcomes. A once-sharp cardiac AI for arrhythmia could start missing critical cardiac events. For a health system CIO, this goes way beyond a technical glitch. It’s a core operational and clinical risk that destroys clinician trust in AI and can bring on serious financial and reputational damage. The real work is building the machinery to catch this drift and allow for fast, regulated model updates without drowning in paperwork for every little change.

Pioneering Continuous Monitoring: Lessons from Duke and Mayo Clinic

Fixing algorithmic drift means you can’t just ‘deploy and forget’ your models. You have to switch to active, continuous performance auditing and governance. A couple of the leaders figuring this out are Duke University Health System and Mayo Clinic, both founding members of the Coalition for Health AI (CHAI). They treat model deployment as the beginning of the journey, not the end. At Duke, for instance, a multi-disciplinary AI governance committee is in charge of the entire lifecycle of their clinical AI. They require continuous monitoring, often by feeding real-time performance stats right back into their EHR. That way, any dip below a performance benchmark gets flagged automatically. For their SaMD (Software as a Medical Device) apps, especially the ones used in high-stakes decisions, Duke’s framework is very specific:

• Performance Dashboards: Dashboards tracking key performance indicators (KPIs) like sensitivity and specificity that are constantly updated and checked against the expected thresholds.
• Adverse Event Reporting: Plugging AI-related adverse event reporting right into the hospital’s existing safety systems, so if a bad outcome might be tied to a degraded model, it gets captured and investigated immediately.
• Retraining Triggers: Hard-coded triggers for model retraining that are automatically pulled when statistical analysis shows the input data has drifted too far or performance has been in a slump.

This whole setup is designed to catch drift early, before it can actually affect patient care. Duke University Health System AI governance protocol Mayo Clinic has built out similar governance which they call ‘Model Maintenance and Evolution’ in their work with CHAI. It’s a very hands-on process that includes:

• Prospective Monitoring: A constant feedback loop where they audit a sample of AI-driven decisions, comparing the model’s prediction to the actual clinical outcome to spot any weirdness.
• Data Integrity Checks: Routine checks on the input data streams to make sure garbage data isn’t getting in and quietly poisoning the model.
• Human-in-the-Loop Oversight: Keeping a human-in-the-loop for the most critical tools, giving clinicians the power to override the AI and provide feedback that helps make the model better.

It’s telling that both Duke and Mayo insist these governance committees aren’t just for data scientists. They’re staffed with clinicians, data scientists, ethicists, and IT security experts to get the full picture. Mayo Clinic clinical AI governance charter

The Role of Regulatory Frameworks: FDA PCCP and CHAI Guidelines

Regulators are getting into the game, too, which gives developers and health systems a much-needed playbook. The FDA’s Predetermined Change Control Plan (PCCP) framework is the main event here. After finalizing guidance on December 4, 2024, and revising it on August 18, 2025, the FDA created a way for AI/ML medical devices to be updated without going back for a new premarket submission every single time. Why does this matter? Because without a PCCP, every time a company retrained its cardiac AI on new patient data to keep it accurate, they’d theoretically have to file for a new 510(k) clearance. That would make continuous improvement completely impossible from a cost and time perspective. The PCCP forces developers to define upfront what kinds of changes they plan to make, how they’ll make them, and what performance guardrails the model has to stay within, creating a sanctioned path for fighting algorithmic drift. FDA guidance on Predetermined Change Control Plans Working alongside the FDA’s efforts, the Coalition for Health AI (CHAI), the group Duke and Mayo Clinic helped found, has put out its own consensus guidelines for using AI responsibly. Their big focus is on transparency, accountability, and continuous performance monitoring. Basically, CHAI is trying to create a shared vocabulary and a set of best practices so that when a hospital is evaluating a new AI tool, they’re all asking the same tough questions. Having both the FDA’s PCCP framework and industry standards from CHAI working together is what will finally give investors the confidence to fund these tools and CIOs the confidence to deploy them safely.

Building a Strong Governance Framework: A Mandate for Health Systems and Investors

So if you’re a health system CIO or a VC partner looking at this space, the message is simple: AI governance isn’t just more compliance paperwork. It’s the only way to manage the very real long-term operational and clinical risks. Pouring money into an AI tool without a rock-solid plan for monitoring performance and managing drift is like installing a new MRI and then never scheduling a service call. It’s going to fail, it’s just a question of when. Any real governance framework has to include:

• Dedicated AI Governance Committee: A dedicated AI governance committee that isn’t just tech people. You need clinicians, ethicists, and lawyers in the room.
• Proactive Monitoring Protocols: Monitoring protocols (both automated and manual) to track model performance against its original baseline and actual patient outcomes.
• Clear Retraining and Update Policies: A clear policy for when and how to retrain models, using pathways like the FDA’s PCCP to stay on the right side of regulators.
• Data Quality Assurance: Constant data quality assurance to make sure the data feeding the model is clean.
• Transparency and Documentation: Rigorous documentation of every model update, performance audit, and any observed clinical impact.

The market is already starting to figure this out. Companies that build their products around GMLP (Good Machine Learning Practice) and have continuous auditing baked in from day one are going to win on trust and have a real competitive edge. For investors, this means your due diligence can’t stop at the 510(k) clearance. You have to dig into the company’s post-market surveillance and drift management plan. What’s their process? Is it real? If a company can’t give you a clear, verifiable answer for how they handle algorithmic drift, they are a hidden liability, no matter how great their initial results look. This perspective is built from studying the actual governance protocols at places like Duke University Health System and Mayo Clinic, and from the guidelines being hammered out by the Coalition for Health AI and the FDA’s PCCP team.