Conference item
Self-supervised co-training for video representation learning
- Abstract:
- The objective of this paper is visual-only self-supervised video representation learning. We make the following contributions: (i) we investigate the benefit of adding semantic-class positives to instance-based Info Noise Contrastive Estimation (InfoNCE) training, showing that this form of supervised contrastive learning leads to a clear improvement in performance; (ii) we propose a novel self-supervised co-training scheme to improve the popular infoNCE loss, exploiting the complementary information from different views, RGB streams and optical flow, of the same data source by using one view to obtain positive class samples for the other; (iii) we thoroughly evaluate the quality of the learnt representation on two different downstream tasks: action recognition and video retrieval. In both cases, the proposed approach demonstrates state-of-the-art or comparable performance with other self-supervised approaches, whilst being significantly more efficient to train, i.e. requiring far less training data to achieve similar performance.
- Publication status:
- Published
- Peer review status:
- Peer reviewed
Actions
Access Document
- Files:
-
-
(Preview, Version of record, pdf, 1.8MB, Terms of use)
-
Authors
- Publisher:
- Neural Information Processing Systems Foundation, Inc.
- Host title:
- Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
- Pages:
- 1-12
- Publication date:
- 2020-12-08
- Acceptance date:
- 2020-09-25
- Event title:
- 34th Conference on Neural Information Processing Systems (NeurIPS)
- Event location:
- Virtual
- Event website:
- https://neurips.cc/
- Event start date:
- 2020-12-06
- Event end date:
- 2020-12-12
- Language:
-
English
- Keywords:
- Pubs id:
-
1140246
- Local pid:
-
pubs:1140246
- Deposit date:
-
2020-11-02
- ARK identifier:
Terms of use
- Copyright date:
- 2020
- Notes:
- This paper was presented at the 34th Conference on Neural Information Processing Systems (NeurIPS), 6-12 December 2020, Virtual. This is the publisher's version of the paper. The final version is available online from the Neural Information Processing Systems Foundation at: https://proceedings.neurips.cc//paper_files/paper/2020/hash/3def184ad8f4755ff269862ea77393dd-Abstract.html
If you are the owner of this record, you can report an update to it here: Report update to this record