Linking reinforcement learning and cognitive maps to understand how the brain represents abstract tasks

Thesis

The terms “reinforcement learning” (RL) and “cognitive maps” are both commonly rooted in early behaviourist psychology. However, in neuroscience, these two subfields have largely progressed in parallel. In this thesis, I use ideas and techniques from one subfield to address open questions in the other.

While the theory of RL has dramatically advanced our understanding of the brain’s learning algorithms, we still don’t know how RL tasks are represented. Here, I take inspiration from the well-studied representations of the cognitive map in spatial tasks to investigate how abstract non-spatial RL tasks are represented and how task knowledge might be generalised to novel situations. I present converging results suggesting that the same areas that encode the structure of spatial tasks also encode the structure of abstract RL tasks. In addition, I use ideas from spatial cognitive maps to suggest novel interpretations of heavily-studied RL neural signals. Further, taking inspiration from the discovery of inference mechanisms over the structure of spatial tasks, I suggest a study, for which I already have a task and a model, that could shed light on similar structural inference mechanisms in RL tasks.

While navigation and planning in physical space have been thoroughly studied, it is not clear how animals can navigate through cognitive maps with arbitrary topology. Here I investigate spatial cognitive maps through the lens of RL formalism, and use ideas from RL to suggest a flexible and efficient planning algorithm that can be used in both spatial environments and environments with arbitrary topology.

This thesis demonstrates the progress that can be made by bringing these two subfields together, and hopefully brings us one step closer to understanding the mechanisms underlying the human capacity for flexible behaviour.

Authors: Baram, A

• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: cognitive maps; reinforcement learning; decision making
• Subjects: Neurosciences; Cognitive neuroscience