Gaussian processes on graphs

Thesis

In the ever-growing field of machine learning research, the use of graphs has recently gathered significant interest for modelling on data with relational structures. Graphs and network-based data now exist ubiquitously in the real world, with examples including social networks, transportation, financial exchanges, and brain networks. Therefore, developing models on graphs is essential to allow users to understand and predict the complex nature observed in everyday phenomena. Currently, there is an abundance of literature on graph neural networks, but limited options are available that are probabilistic and Bayesian. In addressing this issue, we develop a series of Gaussian processes (GPs) for graph data in this thesis. Building GPs on graphs is now more feasible thanks to the emergence of graph signal processing, providing us with the tools to handle graph-structured information and smoothness modelling. The first problem we tackle is predicting the evolution of signals with a multi-output Gaussian process. We use kernels defined from the graph Laplacian with learnable spectral filters to predict with the smoothness level that matches the data. We then turn our focus to semi-supervised classification, designing three models for this task each emphasizing on a particular approach: multi-scale modelling, transductive learning, and sheaf modelling. The first approach provides a novel utilization of wavelets on graphs to fully exploit their ability to capture multi-scale properties in the data. Next, we present a unified definition of kernels on graphs with transductive properties, aiming to utilize the distribution of the full dataset to better inform the prediction. This naturally suits semi-supervised problems on graphs where training and testing nodes are generally connected and available at the same time. Finally, we introduce sheaves as a higher order representation of graphs, to design GPs with stronger separation power by learning additional topological structures. Collectively, this thesis represents not only a valuable contribution to the study of GPs for discrete and non-Euclidean data, but also useful alternatives to the more broadly used graph neural networks.

Authors: Zhi, Y-C

• DOI: 10.5287/ora-zgan0ep5x
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: graph predictions; graph signal processing; kernels; semi-supervised learning; gaussian processes
• Subjects: Gaussian processes; Supervised learning (Machine learning); Kernel functions; Signal processing; Machine learning