DRS-VPT: directly relocalizing in a scan with vision point transformers

Conference item

We present DRS-VPT, a feed-forward transformer architecture for foundational image-to-scan registration. Given query images and a reference 3D point cloud, the model predicts the scan pose and point map alongside the poses and point maps of each camera, all expressed in the first camera’s frame. It additionally predicts a coarse-tofine pyramid of per-point and per-pixel features for direct reprojective alignment of the scan to the first image. This formulation unifies downstream tasks such as camera–LiDAR calibration in autonomous driving and indoor camera-to-map relocalization. A single DRS-VPT model achieves state-of-the-art performance for image-to-LiDAR registration in autonomous driving, competitive indoor relocalization without training map-specific weights, and strong zero-shot transfer to unseen environments. We also show qualitatively that the model learns complex scanto-image projection properties such as occlusion of back-facing points.

Authors: Fu, LFT; Fallon, M

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Springer Nature
• Host title: Computer Vision – ECCV 2026: 19th European Conference Malmö, Sweden, September 8–12, 2026 Proceedings, Part XLIX
• Volume: 17049
• Pages: 1-19
• Series: Lecture Notes in Computer Science
• Publication date: 2026-09-11
• Acceptance date: 2026-08-17
• Event title: 19th European Conference on Computer Vision (ECCV 2026)
• Event location: Malmö, Sweden
• Event website: https://eccv.ecva.net/Conferences/2026
• Event start date: 2026-09-08
• Event end date: 2026-09-12
• DOI: 10.1007/978-3-032-37362-5_1
• EISSN: 1611-3349
• ISSN: 0302-9743
• EISBN: 9783032373625
• ISBN: 9783032373618
• Language: English
• Keywords: registration and scene modeling; 3D reconstruction