Reading between the lanes: Road layout reconstruction from partially segmented scenes

Conference item

Autonomous vehicles require an accurate and adequate representation of their environment for decision making and planning in real-world driving scenarios. While deep learning methods have come a long way providing accurate semantic segmentation of scenes, they are still limited to pixelwise outputs and do not naturally support high-level reasoning and planning methods that are required for complex road manoeuvres. In contrast, we introduce a hierarchical, graphbased representation, called scene graph, which is reconstructed from a partial, pixel-wise segmentation of an image, and which can be linked to domain knowledge and AI reasoning techniques.

In this work, we use an adapted version of the Earley parser and a learnt probabilistic grammar to generate scene graphs from a set of segmented entities. Scene graphs model the structure of the road using an abstract, logical representation which allows us to link them with background knowledge. As a proof-of-concept we demonstrate how parts of a parsed scene can be inferred and classified beyond labelled examples by using domain knowledge specified in the Highway Code. By generating an interpretable representation of road scenes and linking it to background knowledge, we believe that this approach provides a vital step towards explainable and auditable models for planning and decision making in the context of autonomous driving.

Authors: Kunze, L; Bruls, T; Suleymanov, T; Newman, P

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Institute for Electrical and Electronics Engineers
• Host title: 21st IEEE International Conference on Intelligent Transportation Systems (ITSC 2018)
• Journal: 21st IEEE International Conference on Intelligent Transportation Systems (ITSC 2018
• Publication date: 2018-12-10
• Acceptance date: 2018-07-02
• DOI: 10.1109/ITSC.2018.8569270