SIN-Bench: Tracing Native Evidence Chains in Long-Context Multimodal Scientific Interleaved Literature
Evaluating whether multimodal large language models truly understand long-form scientific papers remains challenging: answer-only metrics and synthetic "Needle-In-A-Haystack" tests often reward answer matching without requiring a causal, evidence-linked reasoning trace in the document. We propose the "Fish-in-the-Ocean" (FITO) paradigm, which requires models to construct explicit cross-modal evidence chains within native scientific documents. To operationalize FITO, we build SIN-Data, a scientific interleaved corpus that preserves the native interleaving of text and figures. On top of it, we construct SIN-Bench with four progressive tasks covering evidence discovery (SIN-Find), hypothesis verification (SIN-Verify), grounded QA (SIN-QA), and evidence-anchored synthesis (SIN-Summary). We further introduce "No Evidence, No Score", scoring predictions when grounded to verifiable anchors and diagnosing evidence quality via matching, relevance, and logic. Experiments on eight MLLMs show that grounding is the primary bottleneck: Gemini-3-pro achieves the best average overall score (0.573), while GPT-5 attains the highest SIN-QA answer accuracy (0.767) but underperforms on evidence-aligned overall scores, exposing a gap between correctness and traceable support.
696f3d285750539a88acc82b
Yiming Ren
696f3d285750539a88acc82c
Junjie Wang
claimed_verified
2026-01-20T09:39:02.612Z
62579c55b98dcaa7e0de285d
https://cdn-avatars.huggingface.co/v1/production/uploads/62579c55b98dcaa7e0de285d/0YUd5nloul_bW9yolDGGo.jpeg
wangjunjie
wanng
user
wanng
696f3d285750539a88acc82d
Yuxin Meng
696f3d285750539a88acc82e
Yihang Shi
admin_assigned
2026-01-20T09:49:01.198Z
67ebb2ac4d4959913fa71a40
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/m0yk6RvxQIXdg_Labhyw7.png
yihang shi
lalalion
user
lalalion
696f3d285750539a88acc82f
Zhiqiang Lin
admin_assigned
2026-01-20T09:49:07.520Z
6863d7b1d399aca67b578bd8
/avatars/e57c400dbf14df387c6c3ced9032f8be.svg
Zhiqiang Lin
LLLinkzc
user
LLLinkzc
696f3d285750539a88acc830
Ruihang Chu
admin_assigned
2026-01-20T09:49:12.992Z
642e3bcb958faf258a40e89c
/avatars/dad142df2217f8eed1f45c9e7287d3ea.svg
Ruihang Chu
Ruihang
user
Ruihang
696f3d285750539a88acc831
Yiran Xu
696f3d285750539a88acc832
Ziming Li
696f3d285750539a88acc833
Yunfei Zhao
696f3d285750539a88acc834
Zihan Wang
696f3d285750539a88acc835
Yu Qiao
696f3d285750539a88acc836
Ruiming Tang
696f3d285750539a88acc837
Minghao Liu
claimed_verified
2026-02-25T17:33:13.335Z
6417d9ea8f689506e7148417
https://cdn-avatars.huggingface.co/v1/production/uploads/6417d9ea8f689506e7148417/bAYcruWNw4WvmuQcGgcwC.jpeg
minghao
Liam-Liu
user
Liam-Liu
696f3d285750539a88acc838
Yujiu Yang
admin_assigned
2026-01-20T09:49:39.610Z
64ca1fe838837b12d5e529b7
/avatars/44a3ad9e59318784ac531993b5f69f6b.svg
Yujiu Yang
Thu-redrobot
user
Thu-redrobot
SIN-Bench: Tracing Native Evidence Chains in Long-Context Multimodal Scientific Interleaved Literature
Evaluating whether multimodal large language models truly understand long-form scientific papers remains challenging: answer-only metrics and synthetic "Needle-In-A-Haystack" tests often reward answer matching without requiring a causal, evidence-linked reasoning trace in the document. We propose the "Fish-in-the-Ocean" (FITO) paradigm, which requires models to construct explicit cross-modal evidence chains within native scientific documents. To operationalize FITO, we build SIN-Data, a scientific interleaved corpus that preserves the native interleaving of text and figures. On top of it, we construct SIN-Bench with four progressive tasks covering evidence discovery (SIN-Find), hypothesis verification (SIN-Verify), grounded QA (SIN-QA), and evidence-anchored synthesis (SIN-Summary). We further introduce "No Evidence, No Score", scoring predictions when grounded to verifiable anchors and diagnosing evidence quality via matching, relevance, and logic. Experiments on eight MLLMs show that grounding is the primary bottleneck: Gemini-3-pro achieves the best average overall score (0.573), while GPT-5 attains the highest SIN-QA answer accuracy (0.767) but underperforms on evidence-aligned overall scores, exposing a gap between correctness and traceable support.
Researchers introduce the Fish-in-the-Ocean paradigm and SIN-Bench dataset to evaluate multimodal language models' ability to reason over scientific documents with evidence chains, revealing a gap between answer accuracy and traceable support.
multimodal large language models scientific papers evidence chains SIN-Data SIN-Bench SIN-Find SIN-Verify SIN-QA SIN-Summary No Evidence No Score