Learning to hash naturally sorts

Conference item

Learning to hash pictures a list-wise sorting problem. Its testing metrics, e.g., mean-average precision, count on a sorted candidate list ordered by pair-wise code similarity. However, scarcely does one train a deep hashing model with the sorted results end-to-end because of the non-differentiable nature of the sorting operation. This inconsistency in the objectives of training and test may lead to sub-optimal performance since the training loss often fails to reflect the actual retrieval metric. In this paper, we tackle this problem by introducing Naturally-Sorted Hashing (NSH). We sort the Hamming distances of samples' hash codes and accordingly gather their latent representations for self-supervised training. Thanks to the recent advances in differentiable sorting approximations, the hash head receives gradients from the sorter so that the hash encoder can be optimized along with the training procedure. Additionally, we describe a novel Sorted Noise-Contrastive Estimation (SortedNCE) loss that selectively picks positive and negative samples for contrastive learning, which allows NSH to mine data semantic relations during training in an unsupervised manner. Our extensive experiments show the proposed NSH model significantly outperforms the existing unsupervised hashing methods on three benchmarked datasets.

Authors: Yu, J; Shen, Y; Wang, M; Zhang, H; Torr, PHS

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: International Joint Conferences on Artificial Intelligence Organization
• Host title: Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
• Pages: 1587-1593
• Publication date: 2022-07-16
• Acceptance date: 2022-04-21
• Event title: 31st International Joint Conferences on Artificial Intelligence Organization (IJCAI 2022)
• Event location: Vienna, Austria
• Event website: https://ijcai-22.org/
• Event start date: 2022-07-23
• Event end date: 2022-07-29
• DOI: 10.24963/ijcai.2022/221
• ISSN: 1045-0823
• ISBN: 9781956792003
• Language: English
• Keywords: computer vision: transfer, low-shot, semi- and un- supervised learning; computer vision: image and video retrieval; FFR; computer vision: representation learning