Visual vocabulary with a semantic twist

Conference item

Successful large scale object instance retrieval systems are typically based on accurate matching of local descriptors, such as SIFT. However, these local descriptors are often not sufficiently distinctive to prevent false correspondences, as they only consider the gradient appearance of the local patch, without being able to “see the big picture”.

We describe a method, SemanticSIFT, which takes account of local image semantic content (such as grass and sky) in matching, and thereby eliminates many false matches. We show that this enhanced descriptor can be employed in standard large scale inverted file systems with the following benefits: improved precision (as false retrievals are suppressed); an almost two-fold speedup in retrieval speed (as posting lists are shorter on average); and, depending on the target application, a 20 % decrease in memory requirements (since unrequired ‘semantic’ words can be removed). Furthermore, we also introduce a fast, and near state of the art, semantic segmentation algorithm.

Quantitative and qualitative results on standard benchmark datasets (Oxford Buildings 5 k and 105 k) demonstrate the effectiveness of our approach.

Authors: Arandjelović, R; Zisserman, A

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Springer Nature
• Host title: Computer Vision -- ACCV 2014, Part I
• Pages: 178-195
• Series: Lecture Notes in Computer Science
• Series number: 9003
• Publication date: 2015-04-16
• Event title: 12th Asian Conference on Computer Vision (ACCV 2014)
• Event location: Singapore
• Event website: https://dblp.org/db/conf/accv/accv2014-1.html
• Event start date: 2014-11-01
• Event end date: 2014-11-05
• DOI: 10.1007/978-3-319-16865-4_12
• EISSN: 1611-3349
• ISBN: 9783319168647
• Language: English