Self-supervised monocular depth and pose estimation for endoscopy with latent priors

Journal article

Accurate 3D reconstruction in endoscopy enables quantitative and holistic lesion characterization within the gastrointestinal (GI) tract. To achieve this, reliable depth and pose estimation is required. However, endoscopy systems are monocular, and existing methods relying on synthetic datasets or complex models often lack generalizability in challenging endoscopic conditions. We propose a robust self-supervised monocular depth and pose estimation framework that incorporates a StyleGAN-based generator and a Variational Autoencoder (VAE). The StyleGAN generator leverages extensive depth scenes from natural images to condition the depth network, enhancing realism and robustness of depth predictions through latent feature priors. For pose estimation, we reformulate it within a VAE framework, treating pose transitions as latent variables to regularize scale, stabilize z-axis prominence, and improve x-y sensitivity. To further enhance pose stability and generalizability, we introduce a prior transfer module that distills motion knowledge from natural scene SLAM systems. Specifically, pose priors from a pretrained SLAM model-supervised on large-scale natural scene datasets-are used to guide the latent distribution of pose through a KL-divergence reparameterization. This mechanism effectively transfers structural motion priors into the endoscopic domain, improving trajectory consistency under challenging conditions. This dual refinement pipeline enables accurate depth and pose predictions, effectively addressing the GI tract's complex textures and lighting. Extensive evaluations on SimCol, C3VD, and EndoSLAM datasets confirm our framework's superior performance over published self-supervised methods in endoscopic depth and pose estimation. All data descriptions and code are available at https://github.com/EricXuziang/ Self-supervised-with-Latent-Priors.git.

Authors: Xu, Z; Li, B; Hu, Y; Zhang, C; East, J

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: IEEE
• Journal: IEEE Transactions on Medical Imaging
• Place of publication: United States
• Publication date: 2026-03-09
• DOI: 10.1109/tmi.2026.3671423
• EISSN: 1558-254X
• ISSN: 0278-0062
• Pmid: 41801778
• Language: English
• Keywords: cameras; self-supervised learning; deep learning; endoscopy; monocular depth and pose estimation; endoscopes; pose estimation; depth measurement; three-dimensional displays; generators; accuracy; lesions; image reconstruction; colonoscopy