Understanding convolutional neural networks

Thesis

In the past decade, deep learning has fueled a number of exciting developments in artificial intelligence (AI). However, as deep learning is increasingly being applied to high-impact domains, like medical diagnosis or autonomous driving, the impact of its failures also increases. Because of their high complexity (i.e. they are typically composed of millions of parameters), deep learning models are difficult to interpret. Thus, there is a great need for tools that help us understand how such models make their decisions. In this thesis, we introduce several methods for understanding convolutional neural networks (CNNs), the class of deep learning models that is typically applied to visual data — i.e. images and videos. Our techniques span three approaches to understanding a model: 1. describing the relationship between its inputs and outputs; 2. characterizing correlations between a model’s inputs and its internal representation; and 3. using visualization tools to easily and efficiently explore aspects of a model.

First, we tackle the attribution problem of identifying the parts of a model’s input (i.e. image regions) that are most responsible for its output decision. We present two techniques — meaningful perturbations and extremal perturbations – which work by perturbing the input image and learning the regions that when edited out, most affect the model’s prediction. Second, we seek to understand how semantic concepts, from different kinds of textures to various kinds of objects, are recognized by network parameters (a.k.a. neurons). We introduce Net2Vec, a novel paradigm that reveals how combinations of internal neurons encode specific concepts. Lastly, similar to how a stethoscope is used to explore the internal behavior of different parts of the body, we introduce a novel visualization technique — interactive similarity overlays — that allows an AI researcher or developer to quickly and easily explore the internal representation of a model. Together, these methods enable us to scientifically understand the external behavior of CNNs as well as their inner workings.

Authors: Fong, R

• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: explainable AI; interpretability; artificial intelligence; machine learning; computer vision
• Subjects: Computer vision; Neural networks (Computer science); Machine learning