Conference item
CHAOS: chart analysis with outlier samples
- Abstract:
- Charts play a critical role in data analysis and visualization, yet real-world applications often present charts with challenging or noisy features. However, "outlier charts" pose a substantial challenge even for Multimodal Large Language Models (MLLMs), which can struggle to interpret perturbed charts. In this work, we introduce CHAOS (CHart Analysis with Outlier Samples), a robustness benchmark to evaluate MLLMs against chart perturbations systematically. CHAOS encompasses five types of textual and ten types of visual perturbations, each presented at three levels of severity (easy, mid, hard), inspired by the study result of human evaluation. The benchmark includes 13 state-of-the-art MLLMs divided into 3 groups according to the training scope and data. Comprehensive analysis involves two downstream tasks. Extensive experiments and case studies highlight critical insights into the robustness of models across chart perturbations, aiming to guide future research in chart understanding domain. Data and code are publicly available at http: //huggingface.co/datasets/omoured/CHAOS.
- Publication status:
- Published
- Peer review status:
- Peer reviewed
Actions
Access Document
- Files:
-
-
(Preview, Accepted manuscript, pdf, 12.8MB, Terms of use)
-
- Publisher copy:
- 10.1109/icassp55912.2026.11463894
Authors
- Publisher:
- IEEE
- Host title:
- ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Pages:
- 9392-9396
- Publication date:
- 2026-04-21
- Acceptance date:
- 2026-04-21
- Event title:
- IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2026)
- Event location:
- Barcelona, Spain
- Event website:
- https://2026.ieeeicassp.org/
- Event start date:
- 2026-05-03
- Event end date:
- 2026-05-08
- DOI:
- EISSN:
-
2379-190X
- ISSN:
-
1520-6149
- EISBN:
- 9798331567019
- ISBN:
- 9798331567026
- Language:
-
English
- Keywords:
- Pubs id:
-
2414884
- Local pid:
-
pubs:2414884
- Deposit date:
-
2026-06-15
- ARK identifier:
Terms of use
- Copyright holder:
- IEEE
- Copyright date:
- 2026
- Rights statement:
- Copyright © 2026, IEEE
- Notes:
- The author accepted manuscript (AAM) of this paper has been made available under the University of Oxford's Open Access Publications Policy, and a CC BY public copyright licence has been applied.
- Licence:
- CC Attribution (CC BY)
If you are the owner of this record, you can report an update to it here: Report update to this record