Deep geometry-prior for absolute pose regression

Thesis

Camera relocalisation, a core problem in computer vision and robotics, is crucial for transformative technologies such as augmented reality, autonomous navigation, and 3D scene reconstruction. This thesis seeks to push the boundaries of visual relocalisation by studying various methods of integrating geometric priors into the end-to-end deep learning models, addressing three key aspects: training, inference, and network architecture. By harnessing innovative methods, including Neural Radiance Fields and 3D Gaussian Splatting, alongside map-relative pose regression, the thesis contributes to more precise, efficient, and scalable solutions for camera relocalisation.

The research introduces novel training paradigms that incorporate implicit 3Dbased direct photometric and feature-metric matching to absolute pose regression (APR) models. Techniques such as Direct-PoseNet and DFNet enhance APR performance through differentiable rendering.

The investigation then shifts to integrating 3D geometry at inference time, proposing advanced post-processing methods like neural feature synthesis-based pose refinement, uncertainty-aware hierarchical pose refinement, and efficient pose refinement using 3D Gaussians and 3D foundation models. These frameworks demonstrate significant improvements in pose estimation accuracy across various benchmarks.

Finally, the thesis introduces a geometrically informed network architectural design for APR, a map-relative pose regression framework that bridges the gap between end-to-end deep networks with 3D structure-based methods, enabling scalable and robust relocalisation in dynamic and unvisited environments.

Authors: Chen, S

• DOI: 10.5287/ora-go9rkw1z1
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: novel view synthesis; semi-supervised learning; 3D Gaussians splatting; direct matching; absolute pose regression; scene coordinate regression; nerf; camera relocalisation
• Subjects: Computer Vision