Decoding non-invasive brain activity with novel deep-learning approaches

Thesis

This thesis delves into the world of non-invasive electrophysiological brain signals like electroencephalography (EEG) and magnetoencephalography (MEG), focusing on modelling and decoding such data. The research aims to investigate what happens in the brain when we perceive visual stimuli or engage in covert speech (inner speech) and enhance the decoding performance of such stimuli. The findings have significant implications for the development of brain-computer interfaces (BCIs), leading to assistive communication technologies for paralysed individuals. The thesis is divided into two main sections, methodological and experimental work. A central concern in both sections is the large variability present in electrophysiological recordings, whether it be within-subject or between-subject variability, and to a certain extent between-dataset variability.

In the methodological sections, we explore the potential of deep learning for brain decoding. The research acknowledges the urgent need for more sophisticated models and larger datasets to improve the decoding and modelling of EEG and MEG signals. We present advancements in decoding visual stimuli using linear models at the individual subject level. We then explore how deep learning techniques can be employed for group decoding, introducing new methods to deal with between-subject variability. Finally, we also explores novel forecasting models of MEG data based on convolutional and Transformer-based architectures. In particular, Transformer-based models demonstrate superior capabilities in generating signals that closely match real brain data, thereby enhancing the accuracy and reliability of modelling the brain’s electrophysiology.

In the experimental section, we present a unique dataset containing high-trial inner speech EEG, MEG, and preliminary optically pumped magnetometer (OPM) data. We highlight the limitations of current BCI systems used for communication, which are either invasive or extremely slow. While inner speech decoding from non-invasive brain signals has great promise, it has been a challenging goal in the field with limited decoding approaches, indicating a significant gap that needs to be addressed. Our aim is to investigate different types of inner speech and push decoding performance by collecting a high number of trials and sessions from a few participants. However, the decoding results are found to be mostly negative, underscoring the difficulty of decoding inner speech.

In conclusion, this thesis provides valuable insight into the challenges and potential solutions in the field of electrophysiology, particularly in the decoding of visual stimuli and inner speech. The findings could pave the way for future research and advancements in the field, ultimately improving communication capabilities for paralysed individuals.

Authors: Csaky, R

• DOI: 10.5287/ora-zbxbrp778
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: MEG; decoding; EEG; magnetoencephalpography; OPM; neuroimaging; forecasting; electrophysiology; BCI; variability; deep learning; non-invasive
• Subjects: Neural networks (Computer science)