Conference item icon

Conference item

Self-supervised co-training for video representation learning

Abstract:
The objective of this paper is visual-only self-supervised video representation learning. We make the following contributions: (i) we investigate the benefit of adding semantic-class positives to instance-based Info Noise Contrastive Estimation (InfoNCE) training, showing that this form of supervised contrastive learning leads to a clear improvement in performance; (ii) we propose a novel self-supervised co-training scheme to improve the popular infoNCE loss, exploiting the complementary information from different views, RGB streams and optical flow, of the same data source by using one view to obtain positive class samples for the other; (iii) we thoroughly evaluate the quality of the learnt representation on two different downstream tasks: action recognition and video retrieval. In both cases, the proposed approach demonstrates state-of-the-art or comparable performance with other self-supervised approaches, whilst being significantly more efficient to train, i.e. requiring far less training data to achieve similar performance.
Publication status:
Published
Peer review status:
Peer reviewed

Actions

Authors

More by this author
Institution:
University of Oxford
Division:
MPLS
Department:
Engineering Science
Role:
Author
More by this author
Institution:
University of Oxford
Division:
MPLS
Department:
Engineering Science
Role:
Author
More by this author
Institution:
University of Oxford
Division:
MPLS
Department:
Engineering Science
Role:
Author
ORCID:
0000-0002-8945-8573


Engineering & Physical Sciences Research Council More from this funder
Grant:
EP/M013774/1


Publisher:
Neural Information Processing Systems Foundation, Inc.
Host title:
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
Pages:
1-12
Publication date:
2020-12-08
Acceptance date:
2020-09-25
Event title:
34th Conference on Neural Information Processing Systems (NeurIPS)
Event location:
Virtual
Event website:
https://neurips.cc/
Event start date:
2020-12-06
Event end date:
2020-12-12


Language:
English
Keywords:
Pubs id:
1140246
Local pid:
pubs:1140246
Deposit date:
2020-11-02
ARK identifier:

Terms of use


Views and Downloads

Views and downloads will return soon






If you are the owner of this record, you can report an update to it here: Report update to this record

TO TOP