Associative memories via predictive coding

Conference item

Associative memories in the brain receive and store patterns of activity registered by the sensory neurons, and are able to retrieve them when necessary. Due to their importance in human intelligence, computational models of associative memories have been developed for several decades now. In this paper, we present a novel neural model for realizing associative memories, which is based on a hierarchical generative network that receives external stimuli via sensory neurons. It is trained using predictive coding, an error-based learning algorithm inspired by information processing in the cortex. To test the model's capabilities, we perform multiple retrieval experiments from both corrupted and incomplete data points. In an extensive comparison, we show that this new model outperforms in retrieval accuracy and robustness popular associative memory models, such as autoencoders trained via backpropagation, and modern Hopfield networks. In particular, in completing partial data points, our model achieves remarkable results on natural image datasets, such as ImageNet, with a surprisingly high accuracy, even when only a tiny fraction of pixels of the original images is presented. Our model provides a plausible framework to study learning and retrieval of memories in the brain, as it closely mimics the behavior of the hippocampus as a memory index and generative model.

Authors: Salvatori, T; Song, Y; Hong, Y; Sha, L; Frieder, S

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Curran Associates
• Host title: Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
• Volume: 34
• Pages: 3874-3886
• Publication date: 2021-12-06
• Acceptance date: 2021-09-28
• Event title: 35th Conference on Neural Information Processing Systems (NeurIPS 2021)
• Event location: Virtual event
• Event website: https://nips.cc/Conferences/2021/
• Event start date: 2021-12-06
• Event end date: 2021-12-14
• ISBN: 9781713845393
• Language: English
• Keywords: FFR