Constrained video face clustering using 1NN relations

Conference item

In this work, we introduce the Constrained first nearest neighbour Clustering (C1C) method for video face clustering. Using the premise that the first nearest neighbour (1NN) of an instance is sufficient to discover large chains and groupings, C1C builds upon the hierarchical clustering method FINCH by imposing must-link and cannot-link constraints acquired in a self-supervised manner. We show that adding these constraints leads to performance improvements with low computational cost. C1C is easily scalable and does not require any training. Additionally, we introduce a new Friends dataset for evaluating the performance of face clustering algorithms. Given that most video datasets for face clustering are saturated or emphasize only the main characters, the Friends dataset is larger, contains identities for several main and secondary characters, and tackles more challenging cases as it labels also the ‘back of the head’. We evaluate C1C on the Big Bang Theory, Buffy, and Sherlock datasets for video face clustering, and show that it achieves the new state of the art whilst setting the baseline on Friends.

Authors: Kalogeiton, V; Zisserman, A

• Publication status: Published
• Peer review status: Peer reviewed
• Acceptance date: 2020-07-29
• Event title: British Machine Vision Conference 2020
• Event location: Online
• Event website: https://www.bmvc2020-conference.com/
• Event start date: 2020-09-07
• Event end date: 2020-09-10
• Commissioning body: British Machine Vision Association
• Language: English
• Keywords: FFR