Unsupervised multi-object segmentation by predicting probable motion patterns

Internet publication

We propose a new approach to learn to segment multiple image objects without manual supervision. The method can extract objects form still images, but uses videos for supervision. While prior works have considered motion for segmentation, a key insight is that, while motion can be used to identify objects, not all objects are necessarily in motion: the absence of motion does not imply the absence of objects. Hence, our model learns to predict image regions that are likely to contain motion patterns characteristic of objects moving rigidly. It does not predict specific motion, which cannot be done unambiguously from a still image, but a distribution of possible motions, which includes the possibility that an object does not move at all. We demonstrate the advantage of this approach over its deterministic counterpart and show state-of-the-art unsupervised object segmentation performance on simulated and real-world benchmarks, surpassing methods that use motion even at test time. As our approach is applicable to variety of network architectures that segment the scenes, we also apply it to existing image reconstruction-based models showing drastic improvement.

Authors: Karazija, L; Choudhury, S; Laina, I; Rupprecht, C; Vedaldi, A

• Publication status: Published
• Peer review status: Not peer reviewed
• Host title: arXiv
• Publication date: 2022-10-21
• DOI: 10.48550/arxiv.2210.12148
• Language: English