Learning deep neural networks: necessity and scope of prior knowledge, raw data, and labels

Thesis

The recent rise in machine learning has been largely made possible by novel algorithms, such as convolutional neural networks and large-scale labelled datasets. Yet obtaining labelled datasets is expensive, does not scale well, and should not be necessary for learning general representations of data such as images or videos. Thus, by shifting the training of neural networks to not require labelled data, it is possible to obtain more robust, diverse and generelizeable neural networks that can scale to the vast quantities of readily available unlabelled data. Learning transferable representations from raw data can thus drastically reduce the cost of solving new tasks and improve performance in many settings where supervisory signals are scarce. The field of self-supervised learning has therefore become an increasingly popular framework for learning without labels and in this thesis I provide several works that shed light on the workings of self-supervised learning, and that contributed to, and shaped the state of the art in this field. This thesis covers works that: i) analyze the role of image transformations; ii) develop methods for self-supervised clustering in the image and video domain; iii) develop novel representation learning methods for video-audio and video-text data and; iv) propose a new dataset better suited to self-supervised pretraining than the current standard. Put together, these works investigate and further the state of self-supervised learning from multiple dimensions and its insights continue to shape the future of deep learning without labels.

Authors: Asano, YM

• DOI: 10.5287/ora-dp6ddvkjr
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: computer vision; self-supervised learning; representation learning; multi-modal learning
• Subjects: Computer vision