Audio-visual deep learning

Thesis

Human perception and learning are inherently multimodal: we interface with the world through multiple sensory streams, including vision, audition, touch, olfaction and taste. By contrast, automatic approaches for machine perception and learning have traditionally depended on single modalities, by processing, for instance, video, audio or speech separately. The goal of this thesis is instead utilizing the natural co-occurrence of audio and visual information in videos to learn useful tasks....

Expand abstract

Authors: Afouras, T

• DOI: 10.5287/ora-449jbornx
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Subjects: deep learning; machine learning; audio-visual learning; sign language recognition; lip reading; computer vision; sound source localization; audio-visual speech enhancement