Multimodal Fact-Level Attribution for Verifiable Reasoning
Multimodal large language models (MLLMs) are increasingly used for real-world tasks involving multi-step reasoning and long-form generation, where reliability requires grounding model outputs in heterogeneous input sources and verifying individual factual claims. However, existing multimodal grounding benchmarks and evaluation methods focus on simplified, observation-based scenarios or limited modalities and fail to assess attribution in complex multimodal reasoning. We introduce MuRGAt (Multimodal Reasoning with Grounded Attribution), a benchmark for evaluating fact-level multimodal attribution in settings that require reasoning beyond direct observation. Given inputs spanning video, audio, and other modalities, MuRGAt requires models to generate answers with explicit reasoning and precise citations, where each citation specifies both modality and temporal segments. To enable reliable assessment, we introduce an automatic evaluation framework that strongly correlates with human judgments. Benchmarking with human and automated scores reveals that even strong MLLMs frequently hallucinate citations despite correct reasoning. Moreover, we observe a key trade-off: increasing reasoning depth or enforcing structured grounding often degrades accuracy, highlighting a significant gap between internal reasoning and verifiable attribution.
698e90fecace060ff123ac97
David Wan
698e90fecace060ff123ac98
Han Wang
claimed_verified
2026-02-13T09:36:30.556Z
617df9bb402d4d8f8eee3737
https://cdn-avatars.huggingface.co/v1/production/uploads/1635645820205-noauth.jpeg
Han Wang
HanNight
user
HanNight
698e90fecace060ff123ac99
Ziyang Wang
698e90fecace060ff123ac9a
Elias Stengel-Eskin
698e90fecace060ff123ac9b
Hyunji Lee
698e90fecace060ff123ac9c
Mohit Bansal
Multimodal Fact-Level Attribution for Verifiable Reasoning
Multimodal large language models (MLLMs) are increasingly used for real-world tasks involving multi-step reasoning and long-form generation, where reliability requires grounding model outputs in heterogeneous input sources and verifying individual factual claims. However, existing multimodal grounding benchmarks and evaluation methods focus on simplified, observation-based scenarios or limited modalities and fail to assess attribution in complex multimodal reasoning. We introduce MuRGAt (Multimodal Reasoning with Grounded Attribution), a benchmark for evaluating fact-level multimodal attribution in settings that require reasoning beyond direct observation. Given inputs spanning video, audio, and other modalities, MuRGAt requires models to generate answers with explicit reasoning and precise citations, where each citation specifies both modality and temporal segments. To enable reliable assessment, we introduce an automatic evaluation framework that strongly correlates with human judgments. Benchmarking with human and automated scores reveals that even strong MLLMs frequently hallucinate citations despite correct reasoning. Moreover, we observe a key trade-off: increasing reasoning depth or enforcing structured grounding often degrades accuracy, highlighting a significant gap between internal reasoning and verifiable attribution.
MuRGAt is a benchmark for evaluating fact-level multimodal attribution in complex reasoning tasks, requiring models to provide precise citations for their answers across video, audio, and other modalities.
Multimodal large language models multimodal grounding multimodal reasoning fact-level attribution automatic evaluation framework hallucination structured grounding