Data synthesis for downstream tasks in autonomous driving

Thesis

Autonomous vehicles have complex data requirements, and data synthesis has emerged as a solution to alleviate data scarcity. This manuscript presents a set of approaches to tackling this scarcity when training supervised downstream autonomous visions tasks with 4 concepts in mind - realism, alignment to ground-truth, control and scalability.

The first contribution presented uses a combination between generative adversarial networks and cycle generative adversarial networks to multiply existing image data by adding weather effects and changes in levels of illumination. While the method is successful at improving diversity of data, it does not tackle changes in the structure of the generated images. To overcome this, the second contribution employs scene composition, while maintaining realism and quality of associated ground-truth, using a semi-parametric approach. Since the method operates only on 2D images, it is limited in its ability to reason about complex interactions. Thus, the third publication introduces a two-stage approach that splits the problem of data generation into two distinct steps: one that reasons about scene structure and geometry in a 3D representation, and another that is responsible for the appearance of the scene. The last publication finally tackles the issue of structure and stylistic consistency by extending the previous publication with multi-view data, style maps for appearance cues and auto-regressive training.

Approaches are evaluated in terms of visual quality and alignment of structure with ground-truth, along with select experiments that test the suitability of the data as training data in object detection, semantic segmentation and depth completion tasks.

Authors: Musat, V

• DOI: 10.5287/ora-nynqjegzr
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: synthetic data for robotic vision; domain adaptation for visual perception
• Subjects: Computer vision; Data synthesis for autonomous driving; Machine learning