155511 - The Limitations and Risks of Computational Reductionism in Bowel Cancer Screening.pdf (907.71 kB)
AI and clinical decision making: The limitations and risks of computational reductionism in bowel cancer screening
journal contributionposted on 2023-05-21, 16:38 authored by Saleem AmeenSaleem Ameen, Ming WongMing Wong, Kwang YeeKwang Yee, Paul TurnerPaul Turner
Advances in artificial intelligence in healthcare are frequently promoted as ‘solutions’ to improve the accuracy, safety, and quality of clinical decisions, treatments, and care. Despite some diagnostic success, however, AI systems rely on forms of reductive reasoning and computational determinism that embed problematic assumptions about clinical decision-making and clinical practice. Clinician autonomy, experience, and judgement are reduced to inputs and outputs framed as binary or multi-class classification problems benchmarked against a clinician’s capacity to identify or predict disease states. This paper examines this reductive reasoning in AI systems for colorectal cancer (CRC) to highlight their limitations and risks: (1) in AI systems themselves due to inherent biases in (a) retrospective training datasets and (b) embedded assumptions in underlying AI architectures and algorithms; (2) in the problematic and limited evaluations being conducted on AI systems prior to system integration in clinical practice; and (3) in marginalising socio-technical factors in the context-dependent interactions between clinicians, their patients, and the broader health system. The paper argues that to optimise benefits from AI systems and to avoid negative unintended consequences for clinical decision-making and patient care, there is a need for more nuanced and balanced approaches to AI system deployment and evaluation in CRC.
Publication titleApplied Sciences
Department/SchoolTasmanian School of Medicine
Place of publicationSwitzerland
Rights statement© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution 4.0 International (CC BY 4.0) license, (https://creativecommons.org/licenses/by/4.0/)