Continual learning via probabilistic exchangeable sequence modelling

Journal article

Continual learning (CL) refers to the ability to continuously learn and accumulate new knowledge while retaining useful information from past experiences. Although numerous CL methods have been proposed in recent years, it is not straightforward to deploy them directly to real-world decision-making problems due to their computational cost and lack of uncertainty quantification. To address these issues, we propose CL-BRUNO, a probabilistic, Neural Process-based CL model that performs scalable and tractable Bayesian update and prediction via probabilistic exchangeable sequence modelling. Our proposed approach uses deep-generative models to create a unified Bayesian probabilistic framework capable of handling different types of CL problems such as task- and class-incremental learning by modelling data from different tasks as sequences of exchangeable random variables, allowing users to integrate information across different CL scenarios efficiently using a single model, and give easy-to-interpret probabilistic predictions without the need of training or maintaining separate classifiers. Our approach is able to prevent catastrophic forgetting through distributional and functional regularisation without the need of retaining any previously seen samples, making it appealing to applications where data privacy or storage capacity is of concern. Experiments show that CL-BRUNO outperforms existing methods on both natural image and biomedical data sets, confirming its effectiveness in real-world applications.

Authors: Xing, H; Yau, C

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: TMLR
• Journal: Transactions on Machine Learning Research
• Volume: 2025
• Issue: 10
• Article number: 4714
• Publication date: 2025-10-19
• Acceptance date: 2025-09-28
• EISSN: 2835-8856
• Language: English