Algorithms and modelling for large-scale Bayesian data analysis

Thesis

Bayesian statistics has emerged as a leading paradigm for the analysis of complicated datasets and for reasoning and making predictions under uncertainty. However, the framework faces significant difficulties when it is applied at scale. As datasets grow larger, simulation-based approaches to inference become unviably expensive. As models become more complex, the naïve application of traditional inference methods does not always produce valid predictions. And as Bayesian methods are applied to more complicated phenomena, it is often unclear how even to write down a suitable model on which to perform inference in the first place.

This thesis presents three pieces of work aimed at addressing these problems. We describe a Markov chain Monte Carlo method whose cost per iteration does not necessarily scale linearly with the size of the dataset when applied to Bayesian big-data posteriors. We provide a rigorous analysis of this method including precise conditions under which it yields a performance benefit over standard Metropolis--Hastings. We next provide an asymptotic analysis of nested Monte Carlo schemes that are required for certain complex Bayesian models such as probabilistic programs, along with prescriptions to ensure their consistency under well-specified conditions. Finally, we consider the task of learning models from data automatically using deep generative models. We identify a limitation of normalising flow models, which are defined to be homeomorphisms and so must preserve the topology of the prior. We propose a new family of deep generative models to address this, and demonstrate its benefits empirically across a variety of datasets.

Authors: Cornish, JRM

• DOI: 10.5287/ora-zb54kzoa0
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: Normalising flows; Markov chain Monte Carlo methods; Deep generative modelling; Monte Carlo methods; Bayesian statistics
• Subjects: Machine learning