Conditional random fields as recurrent neural networks

Conference item

Pixel-level labelling tasks, such as semantic segmentation, play a central role in image understanding. Recent approaches have attempted to harness the capabilities of deep learning techniques for image recognition to tackle pixel-level labelling tasks. One central issue in this methodology is the limited capacity of deep learning techniques to delineate visual objects. To solve this problem, we introduce a new form of convolutional neural network that combines the strengths of Convolutional Neural Networks (CNNs) and Conditional Random Fields (CRFs)-based probabilistic graphical modelling. To this end, we formulate Conditional Random Fields with Gaussian pairwise potentials and mean-field approximate inference as Recurrent Neural Networks. This network, called CRF-RNN, is then plugged in as a part of a CNN to obtain a deep network that has desirable properties of both CNNs and CRFs. Importantly, our system fully integrates CRF modelling with CNNs, making it possible to train the whole deep network end-to-end with the usual back-propagation algorithm, avoiding offline post-processing methods for object delineation. We apply the proposed method to the problem of semantic image segmentation, obtaining top results on the challenging Pascal VOC 2012 segmentation benchmark.

Authors: Zheng, S; Jayasumana, S; Romera-Paredes, B; Vineet, V; Su, Z

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: IEEE
• Host title: 2015 IEEE International Conference on Computer Vision (ICCV)
• Pages: 1529-1537
• Publication date: 2016-02-18
• Acceptance date: 2015-09-03
• Event title: 2015 IEEE International Conference on Computer Vision (ICCV 2015)
• Event location: Santiago, Chile
• Event website: https://iccv2015.thecvf.com/
• Event start date: 2015-12-07
• Event end date: 2015-12-13
• DOI: 10.1109/iccv.2015.179
• EISSN: 2380-7504
• ISSN: 1550-5499
• EISBN: 9781467383912
• ISBN: 9781467383905
• Language: English