ExpVid: A Benchmark for Experiment Video Understanding & Reasoning
Multimodal Large Language Models (MLLMs) hold promise for accelerating
scientific discovery by interpreting complex experimental procedures. However,
their true capabilities are poorly understood, as existing benchmarks neglect
the fine-grained and long-horizon nature of authentic laboratory work,
especially in wet-lab settings. To bridge this gap, we introduce ExpVid, the
first benchmark designed to systematically evaluate MLLMs on scientific
experiment videos. Curated from peer-reviewed video publications, ExpVid
features a new three-level task hierarchy that mirrors the scientific process:
(1) Fine-grained Perception of tools, materials, and actions; (2) Procedural
Understanding of step order and completeness; and (3) Scientific Reasoning that
connects the full experiment to its published conclusions. Our vision-centric
annotation pipeline, combining automated generation with multi-disciplinary
expert validation, ensures that tasks require visual grounding. We evaluate 19
leading MLLMs on ExpVid and find that while they excel at coarse-grained
recognition, they struggle with disambiguating fine details, tracking state
changes over time, and linking experimental procedures to scientific outcomes.
Our results reveal a notable performance gap between proprietary and
open-source models, particularly in high-order reasoning. ExpVid not only
provides a diagnostic tool but also charts a roadmap for developing MLLMs
capable of becoming trustworthy partners in scientific experimentation.
68ee35b69b77b5223f666173
Yicheng Xu
claimed_verified
2025-10-14T14:24:10.982Z
682c163fa17480053339f270
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/UQZW2bM9UEtlVHDUq0yel.png
Yicheng Xu
linghan199
user
linghan199
68ee35b69b77b5223f666174
Yue Wu
68ee35b69b77b5223f666175
Jiashuo Yu
claimed_verified
2025-11-26T09:28:36.621Z
64b89a14cf14c2fabe96664c
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/tEd3fBjMcEubF4plqzcUz.jpeg
Jiashuo Yu
awojustin
user
awojustin
68ee35b69b77b5223f666176
Ziang Yan
68ee35b69b77b5223f666177
Tianxiang Jiang
claimed_verified
2025-11-11T19:52:03.575Z
6744754ff9940208b97a6a9a
https://cdn-avatars.huggingface.co/v1/production/uploads/6744754ff9940208b97a6a9a/PRG6_0jAfsj0uoUJvKyWf.png
Tianxiang Jiang
Eurayka
user
Eurayka
68ee35b69b77b5223f666178
Yinan He
68ee35b69b77b5223f666179
Qingsong Zhao
68ee35b69b77b5223f66617a
Kai Chen
68ee35b69b77b5223f66617b
Yu Qiao
68ee35b69b77b5223f66617c
Limin Wang
68ee35b69b77b5223f66617d
Manabu Okumura
68ee35b69b77b5223f66617e
Yi Wang
ExpVid: A Benchmark for Experiment Video Understanding & Reasoning
Multimodal Large Language Models (MLLMs) hold promise for accelerating
scientific discovery by interpreting complex experimental procedures. However,
their true capabilities are poorly understood, as existing benchmarks neglect
the fine-grained and long-horizon nature of authentic laboratory work,
especially in wet-lab settings. To bridge this gap, we introduce ExpVid, the
first benchmark designed to systematically evaluate MLLMs on scientific
experiment videos. Curated from peer-reviewed video publications, ExpVid
features a new three-level task hierarchy that mirrors the scientific process:
(1) Fine-grained Perception of tools, materials, and actions; (2) Procedural
Understanding of step order and completeness; and (3) Scientific Reasoning that
connects the full experiment to its published conclusions. Our vision-centric
annotation pipeline, combining automated generation with multi-disciplinary
expert validation, ensures that tasks require visual grounding. We evaluate 19
leading MLLMs on ExpVid and find that while they excel at coarse-grained
recognition, they struggle with disambiguating fine details, tracking state
changes over time, and linking experimental procedures to scientific outcomes.
Our results reveal a notable performance gap between proprietary and
open-source models, particularly in high-order reasoning. ExpVid not only
provides a diagnostic tool but also charts a roadmap for developing MLLMs
capable of becoming trustworthy partners in scientific experimentation.
ExpVid, a new benchmark for evaluating multimodal large language models on scientific experiment videos, highlights gaps in fine-grained perception, procedural understanding, and scientific reasoning.
Multimodal Large Language Models MLLMs ExpVid benchmark scientific experiment videos three-level task hierarchy fine-grained perception procedural understanding scientific reasoning vision-centric annotation pipeline automated generation multi-disciplinary expert validation coarse-grained recognition fine details state changes high-order reasoning proprietary models open-source models