ELIP: enhanced visual-language foundation models for image retrieval

Conference item

The objective in this paper is to improve the performance of text-to-image retrieval. To this end, we introduce a new framework that can boost the performance of large-scale pre-trained vision-language models, so that they can be used for text-to-image re-ranking. The approach, Enhanced LanguageImage Pre-training (ELIP), uses the text query, via a simple MLP mapping network, to predict a set of visual prompts to condition the ViT image encoding. ELIP can easily be applied to the commonly used CLIP, SigLIP and BLIP-2 networks. On the evaluation side, we set up two new out-of-distribution (OOD) benchmarks, Occluded COCO and ImageNet-R, to assess the zeroshot generalisation of the models to different domains. The results demonstrate that ELIP significantly boosts CLIP/SigLIP/SigLIP2 text-to-image retrieval performance and outperforms BLIP-2 on several benchmarks, as well as providing an easy means to adapt to OOD datasets.

Authors: Zhan, G; Liu, Y; Han, K; Xie, W; Zisserman, A

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: IEEE
• Publication date: 2026-01-20
• Acceptance date: 2025-09-15
• Event title: 22nd Conference on Content-Based Multimedia Indexing (CBMI 2025)
• Event location: Dublin, Ireland
• Event website: https://www.cbmi2025.org/
• Event start date: 2025-10-22
• Event end date: 2025-10-24
• DOI: 10.1109/CBMI66578.2025.11339290
• Language: English
• Keywords: image-text retrieval; vision-language models