Scalable machine learning algorithms using path signatures

Thesis

In this thesis, we consider the integration of path signatures–a mathematical object rooted in rough path theory–with scalable machine learning algorithms to address challenges in sequential and structured data modelling. The key topics considered include:
• Path Signatures: Introduced as a hierarchical and theoretically robust feature representation for sequential data, path signatures faithfully capture dynamics while offering properties like invariance to reparameterization and tree-like equivalence. Challenges like computational overheads are addressed through novel algorithms throughout the thesis.
• Gaussian Processes: We demonstrate embedding signature kernels into Gaussian process models, offering an expressive probabilistic modelling approach for sequential data while tackling computational barriers via sparse variational inference techniques. This approach enhances performance on probabilistic time series classification tasks.
• Seq2Tens Framework: Combines signature features with deep learning, using iterations of low-rank layers to mitigate computational costs while retaining expressiveness. Applications include time series classification, mortality prediction, generative modelling.
• Graph Representation: Extends path signatures to graph data and connects them to hypo-elliptic diffusions, combined with low-rank techniques offering scalable architectures for capturing global and local graph structures, and outperforming conventional graph neural networks on tasks requiring long-range reasoning.
• Random Fourier Signature Features: Introduces scalable random feature-based approximations for signature kernels with supporting theoretical results, overcoming computational limitations for large datasets while retaining state-of-the-art performance.
• Recurrent Sparse Spectrum Signature Gaussian Processes: Combines Random Fourier Signature Features with Gaussian Processes and a forgetting mechanism for adaptive con text focus in time series forecasting, bridging short- and long-term dependencies.

The topics are divided into separate, self-contained chapters that can be read independently.

Authors: Toth, GC

• DOI: 10.5287/ora-eq02avwdr
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: low-rank approximations; kernel learning; deep learning; tensors; stochastic analysis; rough path theory; mathematics; path signatures; graphs; time series; machine learning; concentration of measure; random features
• Subjects: Probability measures; Mathematics; Machine learning; Stochastic analysis