MedMisBench: measuring epistemic resilience of LLMs under misleading medical context

Conference item

Large language models (LLMs) now reach expert-level scores on medical licensing exams, encouraging the assumption that high scores imply safe medical judgment while patients increasingly use them for health advice. We show this assumption is fragile: when misleading context is injected into questions that LLMs originally answer correctly, they abandon the correct answer. We call the ability to maintain correct judgment under adversarial context epistemic resilience, and introduce MedMisBench to measure it. MedMisBench contains 10,932 medical question items and 48,889 misleading context-option pairs spanning medical reasoning and optionbased proxies for patient-journey and agentic settings; MEDMISAGENTBENCH extends it beyond multiple choice with 100 tool-mediated electronic health record (EHR) tasks. Across 11 model configurations, mean accuracy falls from 71.1% on original questions to 38.0% under focused misleading context, with 51.5% attack success. The most damaging injections are formal, rule-like fabrications: authority-framed falsehoods reach 69.5% attack success and exception-poisoning claims reach 64.1%. The effect replicates where the benchmark is most tightly controlled. On 63 clinician-validated item–target pairs, all 11 configurations decline, with mean accuracy falling from 72.4% to 25.5%, and on MEDMISAGENTBENCH agentic task success falls from 85.0% to 54.0% for the strongest configuration. A 14-member clinical panel from 7 countries identified serious potential harm in 34 of 89 reviewed cases (38.2%). MedMisBench exposes a structural blind spot in medical LLM evaluation: existing benchmarks measure what models know, not whether they preserve correct judgment under misleading context.

Authors: Zhou, H; Zou, X; Wu, J; Wu, S; Yu, J

• Publication status: Accepted
• Peer review status: Peer reviewed
• Acceptance date: 2026-09-24
• Event title: 40th Conference on Neural Information Processing Systems (NeurIPS 2026)
• Event location: Sydney, Australia, Atlanta, GA, USA and Paris, France
• Event website: https://neurips.cc/Conferences/2026
• Event start date: 2026-12-06
• Event end date: 2026-12-13
• Language: English