Healthcare AI’s Bias Blind Spot: The Cost-Proxy Problem

The promise of artificial intelligence in healthcare is often framed in terms of efficiency and precision, yet a critical, often overlooked, dimension is its potential to exacerbate existing health inequities. A particularly insidious mechanism through which this occurs is the reliance on cost as a proxy for health needs within algorithmic decision-making. This practice, while seemingly benign on the surface, can embed and amplify racial bias, leading to documented failures in patient care and raising profound questions about the ethical deployment of AI in clinical settings.

The Proxy Variable Problem: When Cost Becomes a Stand-In for Sickness

The core of the problem lies in how certain algorithms are designed to predict healthcare needs. Instead of directly measuring an individual’s health status or severity of illness, some systems use historical healthcare spending as a primary indicator. The assumption is that higher past costs correlate with greater future health needs. However, as research has starkly demonstrated, this assumption is deeply flawed and racially biased.

A landmark study co-authored by Ziad Obermeyer, a physician and researcher affiliated with UC Berkeley and Harvard T.H. Chan School, meticulously uncovered this bias within an algorithm widely used by Optum, a subsidiary of UnitedHealth Group, to identify patients for high-risk care management programs. The algorithm, designed to predict which patients would benefit most from intensive care coordination, used healthcare costs as its primary proxy for health needs. The findings were alarming: at a given risk score, Black patients were significantly sicker than white patients. This meant that for the same predicted level of “need” (based on cost), Black patients had more chronic illnesses and required more intensive medical intervention than their white counterparts. Ziad Obermeyer’s study on algorithmic bias in healthcare

The implication of this design flaw is profound: the algorithm systematically underestimated the health needs of Black patients, channeling fewer resources and less proactive care to those who, by objective clinical measures, were sicker. This isn’t a case of intentional malice but rather a consequence of algorithmic design that leverages a biased proxy variable. Due to systemic inequities in healthcare access, insurance coverage, and historical discrimination, Black patients often incur lower healthcare costs for equivalent or worse health conditions compared to white patients. When an algorithm then uses these lower costs to predict lower future health needs, it perpetuates and amplifies existing disparities.

As Ruha Benjamin, a Princeton University professor whose work explores the social dimensions of science, technology, and medicine, argues, these biases are not merely “bugs” in the system but often “features” reflecting societal inequalities encoded into technological designs. The algorithms, in essence, learn from and reproduce the biases present in the historical data they are trained on. In this context, the relationship “Cost as proxy for health needs is the most widespread mechanism for algorithmic racial bias” becomes undeniably clear. DP03 further elucidates the magnitude of this issue, revealing how widespread these cost-based algorithmic approaches are across healthcare systems.

Regulatory Scrutiny and the Path Forward for Responsible AI

The revelations surrounding such biased algorithms have spurred increased scrutiny from regulatory bodies and academic institutions alike. The FDA SaMD Framework, while primarily focused on the safety and efficacy of Software as a Medical Device, implicitly demands that AI systems do not introduce new harms or exacerbate existing health disparities. An algorithm that systematically undertriages a specific demographic based on a flawed proxy variable clearly falls short of this expectation, posing a direct threat to patient safety.

Similarly, the FTC Algorithmic Fairness guidelines emphasize the need for transparency, accountability, and the prevention of discriminatory outcomes in algorithmic decision-making. The Optum case serves as a stark example of how a lack of rigorous fairness auditing can lead to significant real-world harms. Experts from UC Berkeley, Princeton University, and Harvard T.H. Chan School continue to advocate for robust methods to identify and mitigate these biases, pushing for a paradigm shift from solely focusing on predictive accuracy to prioritizing equitable outcomes.

Responsible AI in healthcare, particularly for Patient Safety Advocates (A5), FDA/Regulatory Officers (A3), and Clinical Informaticists (A2), necessitates a fundamental re-evaluation of the data inputs and proxy variables used in algorithm development. Clinical validation must extend beyond overall performance metrics to specifically assess fairness across diverse demographic groups. This involves moving away from easily quantifiable but biased proxies like cost and towards more direct, clinically relevant measures of health status. It also demands a proactive approach to bias detection and mitigation throughout the AI lifecycle, from data collection and model training to deployment and continuous monitoring. FTC guidance on algorithmic fairness Ignoring these embedded biases risks transforming AI from a tool for progress into a powerful engine for perpetuating and deepening health inequities. Academic research on mitigating algorithmic bias in healthcare