gLSTM: mitigating over-squashing by increasing storage capacity

Conference item : Poster

Graph Neural Networks (GNNs) leverage the graph structure to transmit information between nodes, typically through the message-passing mechanism. While these models have found a wide variety of applications, they are known to suffer from over-squashing, where information from a large receptive field of node representations is collapsed into a single fixed sized vector, resulting in an information bottleneck. In this paper, we re-examine the over-squashing phenomenon through the lens of model storage and retrieval capacity, which we define as the amount of information that can be stored in a node’s representation for later use. We study some of the limitations of existing tasks used to measure over-squashing and introduce a new synthetic task to demonstrate that an information bottleneck can saturate this capacity. Furthermore, we adapt ideas from the sequence modeling literature on associative memories, fast weight programmers, and the xLSTM model to develop a novel GNN architecture with improved capacity. We demonstrate strong performance of this architecture both on our capacity synthetic task, as well as a range of real-world graph benchmarks.

Authors: Blayney, H; Arroyo, A; Dong, X; Bronstein, M

• Publication status: Published
• Peer review status: Peer reviewed
• Host title: Proceedings of the 14th International Conference on Learning Representations (ICLR 2026)
• Article number: 17852
• Publication date: 2026-01-26
• Acceptance date: 2026-01-26
• Event title: 14th International Conference on Learning Representations (ICLR 2026)
• Event location: Rio de Janeiro, Brazil
• Event website: https://iclr.cc/Conferences/2026
• Event start date: 2026-04-23
• Event end date: 2026-04-27
• Language: English
• Keywords: graph neural network; oversquashing; associative memory
• Subtype: Poster