MinkOcc: Towards real-time label-efficient semantic occupancy prediction

Conference item

Developing 3D semantic occupancy prediction models often relies on dense 3D annotations for supervised learning, a process that is both labor and resource-intensive, underscoring the need for label-efficient or even label-free approaches. To address this, we introduce MinkOcc, a multimodal 3D semantic occupancy prediction framework for cameras and LiDARs that proposes a two-step semi-supervised training procedure. Here, a small dataset of explicitly 3D annotations warm-starts the training process; then, the supervision is continued by simpler-to-annotate accumulated LiDAR sweeps and images – semantically labelled through vision foundational models. MinkOcc effectively utilizes these sensor-rich supervisory cues and reduces reliance on manual labeling by 90% while maintaining competitive accuracy. In addition, the proposed model incorporates information from LiDAR and camera data through early fusion and leverages sparse convolution networks for real-time prediction. With its efficiency in both supervision and computation, we aim to extend MinkOcc beyond curated datasets, enabling broader real-world deployment of 3D semantic occupancy prediction in autonomous driving.

Authors: Sze, S; De Martini, D; Kunze, L

• Publication status: Accepted
• Peer review status: Peer reviewed
• Publisher: IEEE
• Acceptance date: 2025-06-30
• Event title: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025)
• Event location: Hangzhao, China
• Event website: https://iros25.org/
• Event start date: 2025-10-19
• Event end date: 2025-10-25
• Language: English