Lip reading in the wild

Conference item

Our aim is to recognise the words being spoken by a talking face, given only the video but not the audio. Existing works in this area have focussed on trying to recognise a small number of utterances in controlled environments (e.g. digits and alphabets), partially due to the shortage of suitable datasets.

We make two novel contributions: first, we develop a pipeline for fully automated large-scale data collection from TV broadcasts. With this we have generated a dataset with over a million word instances, spoken by over a thousand different people; second, we develop CNN architectures that are able to effectively learn and recognize hundreds of words from this large-scale dataset.

We also demonstrate a recognition performance that exceeds the state of the art on a standard public benchmark dataset.

Authors: Chung, J; Zisserman, A

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Springer, Cham
• Host title: Asian Conference on Computer Vision: ACCV 2016: Computer Vision – ACCV 2016
• Journal: Asian Conference on Computer Vision 2016
• Volume: 10112
• Pages: 87-103
• Publication date: 2017-01-01
• Acceptance date: 2016-05-27
• Event location: Taipei
• DOI: 10.1007/978-3-319-54184-6_6
• ISSN: 0302-9743
• ISBN: 9783319541839