Towards useful interpretability for medical imaging

Thesis

Interpretability, in the context of deep learning, is the ability to explain a model to a human. It is proposed as a solution to many tasks including: debugging, improving user trust, finding model bias, and scientific discovery. Hence, there is increasing demand for new interpretability methods, particularly in high-risk scenarios such as in healthcare. However, measuring a method’s ability to explain a model is a complex, or even impossible task. Instead, we propose to measure its usefulness. To measure a method's usefulness, we must first define its specific use. As part of its use we must specify the type of user, e.g., engineers debugging a model, clinicians diagnosing a patient or patients receiving an automated report.

In this thesis, we improve the understanding and evaluation of interpretability methods with specific use cases in mind. First, we demonstrate the importance of understanding the limitations of a method before using it to interpret models. We define and examine three properties of a popular concept-based interpretability method, providing tools to understand when these properties can cause misleading explanations and demonstrate how they affect a melanoma classification task. Next, we propose a novel concept-based interpretability method that requires no labelled concept data and demonstrate how engineers can use it to debug a chest X-ray classification model or detect maliciously implanted trojans in ImageNet models. Finally, we show the importance of human studies in understanding the effects of interpretability on users. We perform a reader study evaluating the effect of explanations on the trust, reliance and performance of sonographers using a prototype-based interpretable model for gestational age estimation. We show that, although explanations are generally assumed to improve trust, they can reduce trust (and performance) if the model explanation does not match the internal decision making of the users.

Interpretable deep learning is an exciting emerging field and, through embracing thorough evaluation and defining clear goals of what explanations aim to achieve, we can further the development of novel, useful methods.

Authors: Nicolson, A

• DOI: 10.5287/ora-xrxejx5rr
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: XAI; medical imaging; interpretability; ultrasound; explainable AI
• Subjects: Fetus--Ultrasonic imaging; Biomedical engineering; Artificial intelligence