Conference item
Segmenting invisible moving objects
- Abstract:
- Biological visual systems are exceptionally good at perceiving objects that undergo changes in appearance, pose, and position. In this paper, we aim to train a computational model with similar functionality to segment the moving objects in videos. We target the challenging cases when objects are ``invisible'' in the RGB video sequence, for example, breaking camouflage, where visual appearance from a static scene can barely provide informative cues, or locating the objects as a whole even under partial occlusion. To this end, we make the following contributions: (i) In order to train a motion segmentation model, we propose a scalable pipeline for generating synthetic training data, significantly reducing the requirements for labour-intensive annotations; (ii) We introduce a dual-head architecture (hybrid of ConvNets and Transformer) that takes a sequence of optical flows as input, and learns to segment the moving objects even when they are partially occluded or stop moving at certain points in videos; (iii) We conduct thorough ablation studies to analyse the critical components in data simulation, and validate the necessity of Transformer layers for aggregating temporal information and for developing object permanence. When evaluating on the MoCA camouflage dataset, the model trained only on synthetic data demonstrates state-of-the-art segmentation performance, even outperforming strong supervised approaches. In addition, we also evaluate on the popular benchmarks DAVIS2016 and SegTrackv2, and show competitive performance despite only processing optical flow.
- Publication status:
- Published
- Peer review status:
- Peer reviewed
Actions
Access Document
- Files:
-
-
(Preview, Accepted manuscript, pdf, 3.6MB, Terms of use)
-
- Publication website:
- https://www.bmvc2021-virtualconference.com/conference/papers/paper_0056.html
Authors
- Publisher:
- British Machine Vision Association
- Article number:
- 56
- Publication date:
- 2021-11-22
- Acceptance date:
- 2021-10-15
- Event title:
- 32nd British Machine Vision Conference (BMVC 2021)
- Event location:
- Virtual event.
- Event website:
- https://www.bmvc2021-virtualconference.com/
- Event start date:
- 2021-11-22
- Event end date:
- 2021-11-25
- Language:
-
English
- Keywords:
- Pubs id:
-
1233013
- Local pid:
-
pubs:1233013
- Deposit date:
-
2022-01-19
- ARK identifier:
Terms of use
- Copyright holder:
- Lamdouar et al.
- Copyright date:
- 2021
- Rights statement:
- © 2021. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms.
- Notes:
- This is the accepted manuscript version of the conference paper. The final version may be available on the publisher website.
If you are the owner of this record, you can report an update to it here: Report update to this record