Discrete representations of continuous data using deep learning and clustering

Thesis

The divide between continuous and discrete data is a fundamental one in computer science and mathematics, as well as related areas such as cognitive science. Historically, most of computing has operated in the discrete domain, but connectionism offers an alternative set of techniques for representing data with continuous vectors, an alterative which has come to the fore with the advent of deep learning over the past decade. This thesis explores techniques for converting continuous, high-dimensional data, of the sort processed so successfully by deep learning, to discrete compact representations, of the sort used by traditional computing. Each of the five main chapters introduces a novel technique that contributes towards this goal, but is also able to be read as a stand-alone piece of research. These techniques fall under deep learning and clustering, and, in keeping with representation learning in general, are mostly, though not entirely, in the unsupervised setting. Some chapters focus on deep learning or clustering separately as a means to form discrete representations of continuous data. Others explore how to combine both deep learning and clustering in a single end-to-end learning system. Such a combination itself involves the interface between continuous and discrete, as deep learning operates on the former, and clustering on the latter.

Being able to bridge the gap between the worlds of continuous and discrete also aligns with the original goal of AI to model human intelligence, as an important part of human cognition is the movement between the worlds of continuous and discrete. Our sensory input is largely continuous, but we represent it with a natural language and reasoning apparatus that is largely discrete. A machine that one day thinks and acts as a human will have to learn to do the same.

Authors: Mahon, L

• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: unsupervised learning; artificial intelligence; deep learning; clustering; concept learning; machine learning
• Subjects: Machine learning; Artificial intelligence