Smooth-AP: Smoothing the path towards large-scale image retrieval

Conference item

Optimising a ranking-based metric, such as Average Precision (AP), is notoriously challenging due to the fact that it is non-differentiable, and hence cannot be optimised directly using gradient-descent methods. To this end, we introduce an objective that optimises instead a smoothed approximation of AP, coined Smooth-AP. Smooth-AP is a plug-and-play objective function that allows for end-to-end training of deep networks with a simple and elegant implementation. We also present an analysis for why directly optimising the ranking based metric of AP offers benefits over other deep metric learning losses.

We apply Smooth-AP to standard retrieval benchmarks: Stanford Online products and VehicleID, and also evaluate on larger-scale datasets: INaturalist for fine-grained category retrieval, and VGGFace2 and IJB-C for face retrieval. In all cases, we improve the performance over the state-of-the-art, especially for larger-scale datasets, thus demonstrating the effectiveness and scalability of Smooth-AP to real-world scenarios.

Authors: Brown, A; Xie, W; Kalogeiton, V; Zisserman, A

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Springer
• Host title: Computer Vision – ECCV 2020. ECCV 2020
• Volume: 12354
• Pages: 677-694
• Series: Lecture Notes in Computer Science
• Publication date: 2020-11-05
• Acceptance date: 2020-07-02
• Event title: Computer Vision – ECCV 2020
• Event location: Virtual Event
• Event website: https://eccv2020.eu/
• Event start date: 2020-08-23
• Event end date: 2020-08-28
• DOI: 10.1007/978-3-030-58545-7_39
• ISSN: 0302-9743
• EISBN: 978-3-030-58545-7
• ISBN: 978-3-030-58544-0
• Language: English
• Keywords: FFR