A general framework for updating belief distributions

Journal article

We propose a framework for general Bayesian inference. We argue that a valid update of a prior belief distribution to a posterior can be made for parameters which are connected to observations through a loss function rather than the traditional likelihood function, which is recovered as a special case.

Modern application areas make it is increasingly challenging for Bayesians to attempt to model the true data generating mechanism. For instance, when the object of interest is low dimensional, such as a mean or median, it is cumbersome to have to achieve this via a complete model for the whole data distribution. More importantly, there are settings where the parameter of interest does not directly index a family of density functions and thus the Bayesian approach to learning about such parameters is currently regarded as problematic.

Our framework uses loss-functions to connect information in the data to functionals of interest. The updating of beliefs then follows from a decision theoretic approach involving cumulative loss functions. Importantly, the procedure coincides with Bayesian updating when a true likelihood is known, yet provides coherent subjective inference in much more general settings. Connections to other inference frameworks are highlighted.

Authors: Holmes, C; Bissiri, P; Walker, S

• Publication status: Accepted
• Peer review status: Peer reviewed
• Publisher: Wiley
• Journal: Journal of the Royal Statistical Society Series B: Statistical Methodology
• Publication date: 2015-01-01
• ISSN: 1467-9868
• Keywords: Gibbs posteriors; loss-likelihood; PAC-Bayes; maximum entropy; generalized estimating equations; decision theory; self-information loss function; loss function; information; Bayesian updating