Influence of the input data on learning deep representations

Thesis

This thesis studies, through the prism of image classification, the influence of the input data on learning deep representations. Indeed, data is abundant but also multifaceted: samples might come from a single or multiple domains, or they can also be more or less labelled. We show in this thesis that methods to learn deep network representations should be designed according to the type of input data at hand.

Our work covers different training data configurations. We start with the multiple visual domains setting. We tackle this challenge by modifying the network architecture with residual adapters, domain specific modules which lead to a high degree of parameter sharing across domains with better transfer learning performance than standard fine-tuning.

Second, we consider the settings where a part of the data is unlabelled. This thesis spans different levels of supervision but, in all cases, we promote the systematic use of self-supervision pre-training. First, we consider the semi-supervised setting with few labels per class for which we propose an alternating optimisation scheme to avoid overfitting the scarse labelled data. We then learn to discover new classes given some known classes by proposing a two-headed network which jointly learns to cluster the unlabelled data and to classify both the labelled and unlabelled samples. To this end, we propose a clustering loss which learns to connect pairs of samples based on their similarity at the feature level. We show that this loss, proposed for novel class discovery, can also be used to perform deep clustering by choosing a nearest neighbours based similarity.

From another perspective, we look at the influence of the data on the deep representations through the lens of saliency. To interpret which parts of the image contribute the most to the data representation, we introduce a unifying saliency framework based on the learning mechanism of convolutional neural networks. This framework can then be used to derive new or existing backpropagation based saliency methods.

Authors: Sylvestre-Alvise Rebuffi

• DOI: 10.5287/ora-54kgkxxqk
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: Weakly-supervised Learning; Artificial Intelligence; Deep Learning; Computer Vision; Machine Learning
• Subjects: Weakly-supervised Learning; Artificial Intelligence; Deep Learning; Computer Vision; Machine Learning