Fast Bayesian coresets via subsampling and quasi-Newton refinement

Conference item

Bayesian coresets approximate a posterior distribution by building a small weighted subset of the data points. Any inference procedure that is too computationally expensive to be run on the full posterior can instead be run inexpensively on the coreset, with results that approximate those on the full data. However, current approaches are limited by either a significant run-time or the need for the user to specify a low-cost approximation to the full posterior. We propose a Bayesian coreset construction algorithm that first selects a uniformly random subset of data, and then optimizes the weights using a novel quasi-Newton method. Our algorithm is a simple to implement, black-box method, that does not require the user to specify a low-cost posterior approximation. It is the first to come with a general high-probability bound on the KL divergence of the output coreset posterior. Experiments demonstrate that our method provides significant improvements in coreset quality against alternatives with comparable construction times, with far less storage cost and user input required.

Authors: Naik, C; Rousseau, JN; Campbell, T

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Curran Associates
• Host title: Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
• Volume: 1
• Pages: 70-83
• Publication date: 2023-04-01
• Acceptance date: 2022-09-15
• Event title: 36th Conference on Neural Information Processing Systems (NeurIPS 2022)
• Event location: New Orleans, USA
• Event website: https://nips.cc/Conferences/2022
• Event start date: 2022-11-28
• Event end date: 2022-12-09
• ISSN: 1049-5258
• EISBN: 9781713873129
• ISBN: 9781713871088
• Language: English
• Keywords: FFR