AI in Healthcare and the Complex Relationship


Most read

Loading Most Ready posts..


    • AI in healthcare enhances diagnostic accuracy, but biased models pose serious risks, as revealed by a University of Michigan study.

    • Clinicians, even with AI explanations, may be misled by biased models, leading to declines in diagnostic performance.

    • The study underscores the need for effective AI explanation tools and interdisciplinary discussions to address biases, ensuring safe implementation in healthcare.

Artificial intelligence (AI) models in healthcare have shown promise in improving diagnostic decisions, but a recent study conducted at the University of Michigan reveals a concerning aspect. While accurate AI models enhance diagnostic accuracy for some demographics, biased models, fueled by skewed medical data, can lead to serious declines in decision-making. Despite efforts by regulators, such as the U.S. Food and Drug Administration (FDA), to ensure the safety and transparency of AI models, a study published in JAMA suggests that clinicians may still be misled, even when provided with AI explanations.

The challenge of biased AI models

The research focused on AI models and explanations related to patients with acute respiratory failure, a condition with complex diagnostic challenges. Clinicians, including hospitalist physicians, nurse practitioners, and physician assistants, were involved in the study. The baseline diagnostic accuracy of clinicians was approximately 73%, emphasizing the difficulty in determining the causes of respiratory failure.

AI Assistance and Diagnostic Accuracy

In the study, clinicians were asked to make treatment recommendations based on their diagnoses for patients with respiratory failure. The research team evaluated the diagnostic accuracy of 457 healthcare professionals with and without assistance from an AI model. Half of the participants received an AI explanation along with the AI model’s decision, while the other half received only the AI decision without an explanation.

The results indicated that clinicians who received assistance from an AI model, even without explanations, experienced a 2.9 percentage point increase in their diagnostic accuracy. When provided with explanations, their accuracy improved by 4.4 percentage points, demonstrating the potential of AI to enhance clinical decision-making.

Unveiling the Pitfalls: Bias in AI ,models

To test whether explanations could help clinicians identify biased or incorrect AI models, the research team intentionally presented clinicians with models that were trained to be biased. For example, a model predicted a high likelihood of pneumonia if the patient was 80 years or older, showcasing potential pitfalls in AI model training.

The findings revealed a significant decline in accuracy when clinicians were exposed to biased AI models. In instances where explanations explicitly highlighted that the AI was relying on non-relevant information, such as associating low bone density in patients over 80 with pneumonia, clinicians did not recover from the decline in performance. This aligns with concerns about users being deceived by AI models, emphasizing the risks associated with biased training data.

The Role of AI Explanations

While AI explanations were shown to improve diagnostic accuracy in general, the study underscores the challenges in developing effective explanation tools. Clinicians need to understand not only the model’s decision but also the underlying reasoning conveyed by the explanation. The study highlights the need for interdisciplinary discussions to refine explanation tools and ensure effective communication between AI models and clinicians.

Implications for Future Research and Implementation

The University of Michigan research team hopes that their study will stimulate more research into the safe implementation of AI-based models in healthcare, considering the diverse demographics of patients. Additionally, the study emphasizes the need for comprehensive medical education surrounding AI and bias. As AI continues to play a pivotal role in healthcare decision-making, addressing these challenges becomes crucial to ensure patient safety and improved outcomes.

The dual nature of AI in healthcare, as both a beneficial tool and a potential source of bias, necessitates ongoing efforts to strike a balance. Regulators, developers, and clinicians must collaboratively work to enhance the transparency of AI models, develop robust explanation tools, and provide thorough education on AI and bias. As the healthcare industry increasingly integrates AI, addressing these challenges is paramount to building trust in AI applications and ensuring that they contribute positively to clinical decision-making.

Disclaimer. The information provided is not trading advice. Cryptopolitan.com holds no liability for any investments made based on the information provided on this page. We strongly recommend independent research and/or consultation with a qualified professional before making any investment decisions.

Share link:

Editah Patrick

Editah is a versatile fintech analyst with a deep understanding of blockchain domains. As much as technology fascinates her, she finds the intersection of both technology and finance mind-blowing. Her particular interest in digital wallets and blockchain aids her audience.

Stay on top of crypto news, get daily updates in your inbox

Related News

Subscribe to CryptoPolitan