Deep learning for time series prediction and decision making over time

Thesis

In this thesis, we develop a collection of state-of-the-art deep learning models for time series forecasting. Primarily focusing on a closer alignment with traditional methods in time series modelling, we adopt three main directions of research -- 1) novel architectures, 2) hybrid models, and 3) feature extraction. Firstly, we propose two new architectures for general one-step-ahead and multi-horizon forecasting. With the Recurrent Neural Filter (RNF), we take a closer look at the relationship between recurrent neural networks and Bayesian filtering, so as to improve representation learning for one-step-ahead forecasts. For multi-horizon forecasting, we propose the Temporal Fusion Transformer (TFT) -- an attention-based model designed to accommodate the full range of inputs present in common problem scenarios. Secondly, we investigate the use of hybrid models to enhance traditional quantitative models with deep learning components -- using domain-specific knowledge can be used to guide neural network training. Through applications in finance (Deep Momentum Networks) and medicine (Disease-Atlas), we demonstrate that hybrid models can effectively improve forecasting performance over pure methods in either category. Finally, we explore the feature learning capabilities of deep neural networks to devise features for general forecasting models. Considering an application in systemic risk management, we devise the Autoencoder Reconstruction Ratio (ARR) -- an indicator to measure the degree of co-movement between asset returns. When fed as an input into a variety of models, we show that the ARR can help to improve short-term predictions of various risk metrics.

On top of improvements in forecasting performance, we also investigate extensions to enable decision support using deep neural networks, by helping users to better understand their data. With Recurrent Marginal Structural Networks (RMSNs), we introduce a general framework to train deep neural networks to learn causal effects over time, using ideas from marginal structural modelling in epidemiology. In addition, we also propose three practical interpretability use-cases for the TFT, demonstrating how attention weights can be analysed to provide insights into temporal dynamics.

Authors: Lim, B

• DOI: 10.5287/ora-er5eeyyvk
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: bayesian filtering; time series forecasting; deep neural networks
• Subjects: deep learning; time series analysis