Neural associative skill memories for safer robotics and modeling human sensorimotor repertoires

Journal article

Modern robots face a challenge shared by biological systems: how to learn and adaptively express multiple sensorimotor skills. A key aspect of this is developing an internal model of expected sensorimotor experiences to detect and react to unexpected events, guiding self-preserving behaviours. Associative Skill Memories (ASMs) address this by linking movement primitives to sensory feedback, but existing implementations rely on hard-coded libraries of individual skills. A key unresolved problem is how a single neural network can learn a repertoire of skills while enabling integrated fault detection and context-aware execution. Here we introduce Neural Associative Skill Memories (Neural ASMs), a framework that utilises self-supervised temporal predictive coding to integrate skill learning and expression using biologically plausible local learning rules. Unlike traditional ASMs, which require explicit skill selection, Neural ASMs implicitly recognise and express skills through contextual inference, enabling fault detection using ‘predictive surprise’ across the entire learned repertoire. Compared to recurrent neural networks trained via backpropagation through time, our model achieves comparable qualitative performance in skill memory expression while using local learning rules and predicts a biologically relevant speed-accuracy trade-off. By integrating fault detection, reactive control, and skill expression into a single energy-based architecture, Neural ASMs contribute to safer, self-preserving robotics and provide a computational lens to study biological sensorimotor learning.

Authors: Mahajan, P; Tang, M; Li, TE; Havoutis, I; Seymour, B

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Massachusetts Institute of Technology Press
• Journal: Neural Computation
• Volume: 38
• Issue: 1
• Pages: 1–27
• Publication date: 2025-11-14
• Acceptance date: 2025-08-24
• DOI: 10.1162/NECO.a.1475
• EISSN: 1530-888X
• ISSN: 0899-7667
• Language: English