- Abstract:
-
The goal of this work is to determine the audio-video synchronisation between mouth motion and speech in a video.
We propose a two-stream ConvNet architecture that enables the mapping between the sound and the mouth images to be trained end-to-end from unlabelled data. The trained network is used to determine the lip-sync error in a video.
We apply the network to two further tasks: active speaker detection and lip reading. On both tasks we set a new state-of-the-art on standard benchmark datasets.
- Publication status:
- Published
- Peer review status:
- Peer reviewed
- Version:
- Accepted manuscript
- Files:
-
-
(pdf, 3.2mb)
-
- Publisher copy:
- 10.1007/978-3-319-54427-4_19
-
Version unsuitable
-
We have not obtained a suitable full-text for a given research output. See this page for more information.
-
Recently completed
-
Sometimes content is held in ORA but is unavailable for a fixed period of time to comply with the policies and wishes of rights holders.
-
Permissions
-
All content made available in ORA should comply with relevant rights, such as copyright. See this page for more information.
-
Clearance
-
Some thesis volumes scanned as part of the digitisation scheme funded by Dr Leonard Polonsky are currently unavailable due to sensitive material or uncleared third-party copyright content. We are attempting to contact authors whose theses are affected.
- Publisher:
- Springer Publisher's website
- Publication date:
- 2017-03-05
- Acceptance date:
- 2016-05-27
- DOI:
- Pubs id:
-
pubs:656453
- URN:
-
uri:6bdd4768-6fbd-40ac-8efc-edca8a0325b3
- UUID:
-
uuid:6bdd4768-6fbd-40ac-8efc-edca8a0325b3
- Local pid:
- pubs:656453
- Copyright holder:
- Springer International Publishing AG
- Copyright date:
- 2017
- Notes:
-
© Springer International Publishing AG 2017
Conference item
Out of time: automated lip sync in the wild
Actions
Access Document
Why is the content I wish to access not available via ORA?
Bibliographic data (the information relating to research outputs) and full-text items (e.g. articles, theses, reports, etc.) arrive in ORA from several different sources. Unfortunately we are not able to make available the full-text for every research output.
Please contact the ORA team (
) if you have queries regarding unavailable content OR if you are aware of a full-text copy we can make available.
) if you have queries regarding unavailable content OR if you are aware of a full-text copy we can make available.
Content may be unavailable for the following four reasons
Alternative access to the full-text
You may be able to access the full-text directly from the publisher's website using the 'Publisher Copy' link in the 'Links & Downloads' box from a research output's ORA record page. This method may require an institutional or individual subscription to the journal/resource.
Authors
Funding
Bibliographic Details
Terms of use
Metrics
Altmetrics
Dimensions
If you are the owner of this record, you can report an update to it here: Report update to this record