When is black-box AI justifiable to use in healthcare?

Journal article

Although it is reasonable and valuable to seek explanations for decisions made by artificial intelligence (AI), it is simply not possible with black-box AI algorithms. However, these algorithms can produce highly beneficial and efficient outputs that could be extremely useful to patients, treating teams, hospitals, and funding bodies. This poses a dilemma: is black-box AI justifiable to use in healthcare? This article analyses the normative reasons that can defend and justify the use of black-box AI in healthcare; this analysis includes, but does not give lexical priority to, explainability. This is pertinent given the current prohibitions of black-box AI in healthcare, such as in Australia. This article defines justifiability as decisions based on robust reasons and thus identifies reasons that can justify the use of black-box AI in healthcare. These include the algorithms’ explainability and accuracy, the seriousness of the decision's consequences, any relevant bias, the context of the decision, and the level of human intervention. We argue that whilst each of these separate considerations is important, only accuracy and reliability are necessary, and to be sufficient, it is likely that some further reasons arising from the nature and context of the decision will be required.

Authors: Prince, S; Savulescu, J

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: SAGE Publications
• Journal: Big Data and Society
• Volume: 12
• Issue: 4
• Article number: 20539517251386037
• Publication date: 2025-11-28
• DOI: 10.1177/20539517251386037
• EISSN: 2053-9517
• ISSN: 2053-9517
• Language: English
• Keywords: explainability; bioethics; transparency; justifiability; Artificial intelligence