Leveraging domain knowledge for self-supervision in scalable robot learning

Thesis

Deep learning in robotics has a data problem.

Over the past decade, deep learning has revolutionised the application of robotics in the real world. From self-driving cars to drones and warehouse applications, deep learning has become an integral tool in deploying robots into complex domains. Despite many advances in training efficiency, operating performance, and inference speeds, there is one key problem that remains in utilising deep models in real-world applications; data efficiency due to the vast quantities of labelled data required for training.

Despite the clear successes that have emerged by using substantial manually annotated datasets to satisfy this need, human labelling, such as drawing bounding boxes around vehicles, is labour intensive, financially expensive and mentally unfulfilling. These datasets are extremely uneconomical and scale poorly to new geographic locations, weather conditions and robotic tasks. Even when deployed in well-labelled domains, trained models may fail when encountering novel data and have no means for self-improvement.

In this thesis we investigate the paradigm of self-supervised learning as a solution to these shortcomings, in which high fidelity training data is generated completely automatically without any human supervision. In particular, our goal is to utilise domain knowledge in expert systems for curating the data necessary to train deep models.

We apply this methodology on real-world tasks with clear commercial applications, such as predicting drivable paths in camera imagery for mapless urban navigation, learning ephemerality masks for robust monocular visual odometry, and learning keypoints for radar odometry and localisation. Not only do we meet or exceed state-of-the-art performance in these crucial tasks, but we do so with no human supervision. In sharing our approaches and ideas from the past four years, we lay the foundations for more efficient and scalable robot learning in the future.

Authors: Barnes, D

• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: Machine Learning; Robotics
• Subjects: Machine Learning; Robotics