Variational methods for probabilistic inference and decision-making

Thesis

Machine learning models have achieved significant success across a wide range of tasks, but their reliability in safety-critical applications remains a concern, particularly when faced with unfamiliar data. This thesis focuses on improving the reliability and uncertainty quantification of neural networks using probabilistic inference. The first contribution of this thesis addresses shortcomings in parameter-space variational inference, which fails to provide reliable uncertainty estimates. We propose a function-space variational inference method that approximates distributions over functions instead of parameters and improves uncertainty quantification under distribution shifts and significantly reduces catastrophic forgetting in continual learning. The second contribution tackles the challenge of specifying meaningful priors for neural networks. We introduce a method for constructing data-driven priors that incorporate domain knowledge, significantly improving uncertainty quantification in computer vision and language modeling tasks and increasing group robustness under subpopulation shifts. The third contribution is a variational formulation of reinforcement learning in infinite-horizon Markov decision processes. This framework allows learning reward functions and dynamic discount factors directly from environmental interactions, offers a probabilistic justification for KL-regularized reinforcement learning, and improves the performance of RL agents in goal-conditioned sequential decision-making tasks. Overall, this thesis presents probabilistically principled, practical methods for making neural networks more reliable and more suitable for deployment in high-stakes, safety-critical environments.

Authors: Rudner, TGJ

• DOI: 10.5287/ora-nzpkmnnqj
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: AI safety; Bayesian deep learning; reinforcement learning; probabilistic machine learning; variational inference; uncertainty quantification
• Subjects: Statistics; Machine learning; Computer Science