Bayesian learning via stochastic gradient langevin dynamics

Journal article

In this paper we propose a new framework for learning from large scale datasets based on iterative learning from small mini-batches. By adding the right amount of noise to a standard stochastic gradient optimization algorithm we show that the iterates will converge to samples from the true posterior distribution as we anneal the stepsize. This seamless transition between optimization and Bayesian posterior sampling provides an inbuilt protection against overfitting. We also propose a practica...

Expand abstract

Authors: Welling, M; Teh, Y

• Journal: Proceedings of the 28th International Conference on Machine Learning, ICML 2011
• Pages: 681-688
• Publication date: 2011-01-01
• Language: English