Quantifying contributions within multimodal fusion for clinical decisions

Conference item

Many medical conditions are diagnosed using a combination of image findings and patient-specific clinical information. While medical images provide valuable diagnostic information, clinical decision-making also relies on additional demographics or clinical data, such as age or sex. Integrating imaging and patient specific clinical data has the potential to improve classification beyond what either modality can achieve alone. We propose a novel contribution-based fusion (CBF) method to combine image-derived measurements with clinical data for disease classification. Unlike many existing multimodal methods that rely on entangled latent representations and focus primarily on predictive performance, our CBF method explicitly quantify the contribution to the final decision of each modality and its specific inputs. We demonstrate our method on the task of screening for Developmental Dysplasia of the Hip (DDH), where patients are classified as normal or abnormal.

Our proposed method achieves comparable or superior classification performance to established multimodal fusion methods, while providing input feature-level (and therefore modality-level) contributions that are reliably associated with the model’s decision-making process. By reporting how much each modality (imaging vs. clinical data) influenced a decision, the proposed method enables direct quantification of how image and clinical data influence the final classification, providing information that may help clinicians trust the model’s predictions.

Authors: Clement, A; Voiculescu, I

• Publication status: Accepted
• Peer review status: Peer reviewed
• Acceptance date: 2026-08-05
• Event title: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2026)
• Event location: Strasbourg, France
• Event website: https://conferences.miccai.org/2026/en/
• Event start date: 2026-09-27
• Event end date: 2026-10-01
• Language: English
• Keywords: contributions; feature-weighting; clinical decisions; explainability