How do train-time pruning dynamics and pruning schedules affect retinal vessel segmentation?

Conference item

Vision Transformers (ViTs) have become a prevalent architecture in modern computer vision due to their strong scaling with foundation models and generalisability under self-supervised learning. However, with their strong expressivity comes quadratic computational complexity with respect to the number of tokens. Many methods seek to reduce the total token count to improve efficiency while maintaining quantitative performance for downstream tasks. Although token pruning has demonstrated promising efficiency gains, its application to dense segmentation remains comparatively under-explored due to its additional challenges. Therefore, we propose a controlled empirical study that isolates the challenging design choices over the FIVES retinal blood vessel segmentation dataset. We find that by exposing models to any traintime pruning substantially improves segmentation performance relative to soft-masking approaches. Furthermore, increasing the frequency of pruning improves segmentation quality, but yields diminishing efficiency improvements due to additional pruning overhead. Code will be provided upon acceptance.

Authors: Wyatt, J; Voiculescu, I

• Publication status: Accepted
• Peer review status: Peer reviewed
• Publisher: Medical Image Computing and Computer Assisted Intervention Society
• Acceptance date: 2026-07-21
• Event title: 29th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2026)
• Event location: Strasbourg, France
• Event website: https://conferences.miccai.org/2026/en/
• Event start date: 2026-09-27
• Event end date: 2026-10-01
• Language: English
• Keywords: token pruning; vision transformers; retinal vessel segmentation; computational efficiencytoken pruning; computational efficiency