Deep transfer learning with Bayesian inference

Thesis

Since the deep learning revolution, a general trend in machine learning literature has been that large, deep models will consistently outperform small, shallow models. This trend, however, comes with the drawback of ever-increasing compute requirements, with many recent state-of-the-art results requiring resources well out of reach of all but the top industry labs. Such issues raise very real concerns with regards to the democratisation of machine learning research, and left unaddressed could ultimately lead to more power and wealth being concentrated in the institutions which are able to invest extremely large sums of money into their AI research programs today.

Transfer learning techniques are a potential solution to these issues, allowing large, general models to be trained once, and then reused in a variety of situations with minimal computation required to adapt them. This work explores novel algorithms and applications of transfer learning in domains as diverse as hierarchical reinforcement learning, generative modeling, and computational social science. Within the hierarchical reinforcement learning domain, we present an algorithm that allows for transfer between options (i.e., temporally abstracted actions) over separate but similar tasks. In the generative modeling domain, we present an algorithm for reusing existing invertible generative models on new data without incurring any extra training cost. Lastly, in the computational social science domain, we show that knowledge can be transferred from human-designed models in order to detect malicious activity targeting a ranking algorithm.

The common thread between all of the algorithms presented in this thesis is that they are inherently Bayesian. We argue that the Bayesian paradigm naturally lends itself to transfer learning applications, in that Bayesian priors can serve as adaptable, general models which can be transformed into task-specific posteriors through the process of inference.

Authors: Gambardella, A

• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: deep learning; Bayesian inference; machine learning; reinforcement learning; probabilistic programming; generative models; simulation-based inference; transfer learning
• Subjects: Machine learning; Reinforcement learning; Inference; Transfer learning (Machine learning)