Advances in fine-grained visual categorization

Thesis

The objective of this work is to improve performance in fine-grained visual categorization (FGVC). In particular, we are interested in the large-scale classification between hundreds of different flower, bird, dog species. FGVC is challenging due to high intra-class variances caused by deformation, view angle, illumination and occlusion, and low inter-class variance since some categories only differ in detail that only experts notice. Applications include field guides, automatic image annotation, one-click shopping app and 3D reconstruction.

At the start, we discuss the importance of foreground segmentation in FGVC, where we focus on the unsupervised segmentation of image training sets into fore- ground and background in order to improve image classification performance. To this end, we introduce a new scalable, alternation-based algorithm for co-segmentation, Bi-CoS, which is simpler than many of its predecessors, and yet has superior performance on standard benchmark image datasets. Next, we extend BiCos to a new model, Tri- CoS, that adds a class-discriminitiveness term directly into the segmentation objective. The new term aims at removing image regions that, although appearing as foreground, do not contribute to the discrimination between classes.

We also propose a model that combines parts alignment and foreground segmentation into a unified convex framework. The model is called Symbiotic in that part discovery/localization is helped by segmentation and, conversely, the segmentation is helped by the detection (e.g. part layout). The joined system improves over what can be achieved with an analogous system that runs segmentation and part-localization independently.

Finally, we built a new flower dataset consisting of 26,798 high quality images collected by ourselves and 187,559 images gathered from existing datasets. The construction of this dataset follows a strict biological taxonomy. We also evaluate the impact of using the Amazon Mechanical Turk (AMT) service for filtering fine-grained data.

Authors: Chai, Y

• Publication date: 2015
• DOI: 10.5287/ora-qy4eraae4
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: Oxford University, UK
• Language: English
• Keywords: computer vision; fine graind visual categorization; image classification; artificial intelligence
• Subjects: Image understanding