How visual representations map to language feature space in multimodal LLMs

Conference item

Effective multimodal reasoning depends on the alignment of visual and linguistic representations, yet the mechanisms by which vision-language models (VLMs) achieve this alignment remain poorly understood. Following the LiMBeR framework, we deliberately maintain a frozen large language model (LLM) and a frozen vision transformer (ViT), connected solely by training a linear adapter during visual instruction tuning. By keeping the language model frozen, we ensure it maintains its original language representations without adaptation to visual data. Consequently, the linear adapter must map visual features directly into the LLM’s existing representational space rather than allowing the language model to develop specialized visual understanding through fine-tuning. Our experimental design uniquely enables the use of pre-trained sparse autoencoders (SAEs) of the LLM as analytical probes. These SAEs remain perfectly aligned with the unchanged language model and serve as a snapshot of the learned language feature-representations. Through systematic analysis of SAE reconstruction error, sparsity patterns, and feature SAE descriptions, we reveal the layer-wise progression through which visual representations gradually align with language feature representations, converging in middle-to-later layers. This suggests a fundamental misalignment between ViT outputs and early LLM layers, raising important questions about whether current adapter-based architectures optimally facilitate crossmodal representation learning.

Authors: Venhof, C; Khakzar, A; Joseph, S; Torr, P; Nanda, N

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Computer Vision and Pattern Recognition
• Article number: P24
• Publication date: 2025-06-11
• Acceptance date: 2025-02-27
• Event title: 4th Explainable AI for Computer Vision (XAI4CV) Workshop (CVPR 2025)
• Event location: Nashville, TN, USA
• Event website: https://xai4cv.github.io/workshop_cvpr25
• Event start date: 2025-06-11
• Event end date: 2025-06-11
• Language: English