Deep learning for communication: emergence, recognition and synthesis

Thesis

Human intelligence is a social phenomenon tightly coupled to the act and process of communication. Ever since the early prehistoric period, humans have been able to communicate amongst themselves at an unprecedented and unparalleled level compared to all other living species. Communication led humans to develop media such as the spoken and written word to effectively convey the meanings of concrete and abstract concepts, and still today a substantial part of human life is spent communicating and sharing information. The scientific study of communication began in Classical Greece with the work of Aristotle and was to evolve through time into the work on information theory by Claude E. Shannon. This work proposes three novel methods for studying the processes of emergence, recognition, synthesis and enhancement of communication, using recent advances in deep learning. The first method investigates the emergence of communication among agents, and introduces a differentiable way of learning communication protocols. The second studies speech recognition in visual verbal communication, and for the first time solves sentence-level lipreading with deep neural networks trained end-to-end. The third and final method proposes a meta-learning approach for sample efficient verbal communication via text-to-speech synthesis. This thesis advances deep learning in these areas, and defines the premises for the creation of novel technologies for the greater good of society.

Authors: Assael, I

• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: lipreading; speech; lipnet; communication; machine learning; deep learning
• Subjects: Machine learning