Generalised variational inference in infinite dimensions

Thesis

This thesis develops a rigorous mathematical theory for variational inference (VI) and generalised variational inference (GVI) when the underlying parameter space is an infinite-dimensional function space. The insights gained from our improved theoretical understanding of VI and GVI allow us to propose novel approaches for solving the VI and GVI optimisation problem via introduction of infinite-dimensional variational parameters or infinite-dimensional gradient descent methods.

Although GVI and in particular VI have previously been employed for function space inference, the infinitedimensional nature of the parameter space has so far been ignored. This has hindered the development of new inference approaches for which a deep understanding of the mathematical structure underpinning VI and GVI is a prerequisite.

This thesis closes this gap by advancing mathematical concepts from infinite-dimensional analysis and probability theory. In particular we use Gaussian random elements in Banach spaces to formalise VI and GVI for function space inference. Consequently, we can access new analytical tools—such as the Wasserstein gradient flow or parameterisations in the space of Gaussian measures—to solve the VI and GVI optimisation problem.

The result is a rigorous theory for GVI in function space and a plethora of competitive novel algorithms for uncertainty quantification such as Gaussian Wasserstein inference, deep repulsive Langevin ensembles and projected Langevin sampling. All methods are rigorously derived from the same GVI objective and our theory unifies several approaches for uncertainty quantification in function space as well as parameter space.

Authors: Wild, VD

• DOI: 10.5287/ora-8nap4okkw
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: Banach spaces; Bayesian statistics; variational inference; nonparametric bayesian statistics; Hilbert spaces; Wasserstein gradient flow; generalised variational inference; optimal transport
• Subjects: Bayesian statistical decision theory; Nonparamteric Statistics; Variational Inference; Statistics