A compositional approach to reading comprehension tasks using the DisCoCirc natural language processing framework

Thesis

The DisCoCirc framework for natural language processing allows the construction of compositional models of text, by combining units for individual words together according to the grammatical structure of the text to be read by the model. The compositional nature of a model can give rise to two things: compositional generalisation - the ability of a model to generalise outside its training distribution by learning compositional rules underpinning the entire data distribution - and compositional understandability - making sense of how the model works by inspecting its modular components in isolation and the processes through which they are combined.

We present a proof-of-concept implementation of a DisCoCirc model on a trapped-ion quantum computer, solving a hand-crafted question answering task. DisCoCirc allows a quantum model to be trained in classical simulation on small instances, while larger instances at test time are evaluated on a quantum computer. This avoids the barren plateau pitfalls of other common QML setups. We find that the accuracy of the models decay by at most 5% when evaluated on the Test set: the model compositionally generalises.

Secondly, we consider how well DisCoCirc models can learn different types of compositional behaviour. We compare both quantum circuit based models, as well as classical neural networks under a series of training paradigms, on a dataset derived from one of the bAbI tasks, extended to test a series of aspects of compositionality. Both architectures score within 5% of one another on the productivity and substitutivity tasks, but differ for the systematicity and overgeneralisation tasks. Overall, we find the neural models are more prone to overfitting the Train data.

Finally, we inspect the trained models, both by comparing them to manuallyconstructed perfect compositional models, and by considering how the model components interact with one another, explaining how the models behave and exhibiting the models’ compositional understandability.

Authors: Duneau, F

• DOI: 10.5287/ora-o0emdd7nr
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: quantum natural language processing; compositionality; natural language processing; compositional interpretability; compositional generalisation; quantum machine learning
• Subjects: Quantum computing; Compositionality (Linguistics); Natural language processing (Computer science)