University of Tasmania
Browse

AI and clinical decision making: The limitations and risks of computational reductionism in bowel cancer screening

Download (907.71 kB)
journal contribution
posted on 2023-05-21, 16:38 authored by Saleem AmeenSaleem Ameen, Ming WongMing Wong, Kwang YeeKwang Yee, Paul TurnerPaul Turner
Advances in artificial intelligence in healthcare are frequently promoted as ‘solutions’ to improve the accuracy, safety, and quality of clinical decisions, treatments, and care. Despite some diagnostic success, however, AI systems rely on forms of reductive reasoning and computational determinism that embed problematic assumptions about clinical decision-making and clinical practice. Clinician autonomy, experience, and judgement are reduced to inputs and outputs framed as binary or multi-class classification problems benchmarked against a clinician’s capacity to identify or predict disease states. This paper examines this reductive reasoning in AI systems for colorectal cancer (CRC) to highlight their limitations and risks: (1) in AI systems themselves due to inherent biases in (a) retrospective training datasets and (b) embedded assumptions in underlying AI architectures and algorithms; (2) in the problematic and limited evaluations being conducted on AI systems prior to system integration in clinical practice; and (3) in marginalising socio-technical factors in the context-dependent interactions between clinicians, their patients, and the broader health system. The paper argues that to optimise benefits from AI systems and to avoid negative unintended consequences for clinical decision-making and patient care, there is a need for more nuanced and balanced approaches to AI system deployment and evaluation in CRC.

History

Publication title

Applied Sciences

Volume

12

Issue

7

Pagination

1-25

ISSN

2076-3417

Department/School

Tasmanian School of Medicine

Publisher

MDPI AG

Place of publication

Switzerland

Rights statement

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution 4.0 International (CC BY 4.0) license, (https://creativecommons.org/licenses/by/4.0/)

Repository Status

  • Open

Socio-economic Objectives

Prevention of human diseases and conditions; Preventive medicine; Artificial intelligence

Usage metrics

    University Of Tasmania

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC