Assessing Risk in Implementing New Artificial Intelligence Triage Tools—How Much Risk is Reasonable in an Already Risky World?

Nord-Bronzyk, A; Savulescu, J; Ballantyne, A; Braunack-Mayer, A; Krishnaswamy, P; Lysaght, T; Ong, MEH; Liu, N; Menikoff, J; Mertens, M; Dunn, M

Journal article

Assessing Risk in Implementing New Artificial Intelligence Triage Tools—How Much Risk is Reasonable in an Already Risky World?

Abstract:: Risk prediction in emergency medicine (EM) holds unique challenges due to issues surrounding urgency, blurry research-practise distinctions, and the high-pressure environment in emergency departments (ED). Artificial intelligence (AI) risk prediction tools have been developed with the aim of streamlining triaging processes and mitigating perennial issues affecting EDs globally, such as overcrowding and delays. The implementation of these tools is complicated by the potential risks associated with over-triage and under-triage, untraceable false positives, as well as the potential for the biases of healthcare professionals toward technology leading to the incorrect usage of such tools. This paper explores risk surrounding these issues in an analysis of a case study involving a machine learning triage tool called the Score for Emergency Risk Prediction (SERP) in Singapore. This tool is used for estimating mortality risk in presentation at the ED. After two successful retrospective studies demonstrating SERP’s strong predictive accuracy, researchers decided that the pre-implementation randomised controlled trial (RCT) would not be feasible due to how the tool interacts with clinical judgement, complicating the blinded arm of the trial. This led them to consider other methods of testing SERP’s real-world capabilities, such as ongoing-evaluation type studies. We discuss the outcomes of a risk–benefit analysis to argue that the proposed implementation strategy is ethically appropriate and aligns with improvement-focused and systemic approaches to implementation, especially the learning health systems framework (LHS) to ensure safety, efficacy, and ongoing learning.

Publication status:: Published

Peer review status:: Peer reviewed

Actions

Email

Email this record

Send the bibliographic details of this record to your email address.

Your Email
Please enter the email address that the record information will be sent to.

-
Your message (optional)
Please add any additional information to be included within the email.
Cite

Cite this record

APA Style

Nord-Bronzyk, A., Savulescu, J., Ballantyne, A., Braunack-Mayer, A., Krishnaswamy, P., Lysaght, T., Ong, M. E. H., Liu, N., Menikoff, J., Mertens, M., & Dunn, M. (2025). Assessing Risk in Implementing New Artificial Intelligence Triage Tools—How Much Risk is Reasonable in an Already Risky World? Asian Bioethics Review, 17(1), 187–205.

MLA Style

Nord-Bronzyk, A., et al. “Assessing Risk in Implementing New Artificial Intelligence Triage Tools—How Much Risk Is Reasonable in an Already Risky World?” Asian Bioethics Review, vol. 17, no. 1, Springer, 2025, pp. 187–205.

Chicago Style

Nord-Bronzyk, A, J Savulescu, A Ballantyne, A Braunack-Mayer, P Krishnaswamy, T Lysaght, MEH Ong, et al. 2025. “Assessing Risk in Implementing New Artificial Intelligence Triage Tools—How Much Risk Is Reasonable in an Already Risky World?” Asian Bioethics Review 17 (1): 187–205.
Share
Print