Hybrid models with deep and invertible features

Conference item

We propose a neural hybrid model consisting of a linear model defined on a set of features computed by a deep, invertible transformation (i.e. a normalizing flow). An attractive property of our model is that both p(features), the density of the features, and p(targets|features), the predictive distribution, can be computed exactly in a single feed-forward pass. We show that our hybrid model, despite the invertibility constraints, achieves similar accuracy to purely predictive models. Yet the generative component remains a good model of the input features despite the hybrid optimization objective. This offers additional capabilities such as detection of out-of-distribution inputs and enabling semi-supervised learning. The availability of the exact joint density p(targets, features) also allows us to compute many quantities readily, making our hybrid model a useful building block for downstream applications of probabilistic deep learning.

Authors: Eric, N; Matsukawa, A; Teh, Y; Gorur, D; Lakshminarayanan, B

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Proceedings of Machine Learning Research
• Host title: Proceedings of Machine Learning Research
• Journal: Proceedings of Machine Learning Research
• Volume: 97
• Pages: 4723-4732
• Publication date: 2019-06-13
• Acceptance date: 2019-04-22
• Keywords: FFR