VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstruction
Evaluating whether Multimodal Large Language Models (MLLMs) genuinely reason about physical dynamics remains challenging. Most existing benchmarks rely on recognition-style protocols such as Visual Question Answering (VQA) and Violation of Expectation (VoE), which can often be answered without committing to an explicit, testable physical hypothesis. We propose VisPhyWorld, an execution-based framework that evaluates physical reasoning by requiring models to generate executable simulator code from visual observations. By producing runnable code, the inferred world representation is directly inspectable, editable, and falsifiable. This separates physical reasoning from rendering. Building on this framework, we introduce VisPhyBench, comprising 209 evaluation scenes derived from 108 physical templates and a systematic protocol that evaluates how well models reconstruct appearance and reproduce physically plausible motion. Our pipeline produces valid reconstructed videos in 97.7% on the benchmark. Experiments show that while state-of-the-art MLLMs achieve strong semantic scene understanding, they struggle to accurately infer physical parameters and to simulate consistent physical dynamics.
6993ffc550fb2c0be4783dcf
Jiarong Liang
claimed_verified
2026-02-19T09:53:48.327Z
68cccfe16e4618473d571b1f
https://cdn-avatars.huggingface.co/v1/production/uploads/68cccfe16e4618473d571b1f/xeF5Ojc5Q1qvtnWy7XoQL.jpeg
Jiarong Liang
lllqaq
user
lllqaq
6993ffc550fb2c0be4783dd0
Max Ku
claimed_verified
2026-02-18T09:07:10.293Z
631d760344503b7227837242
https://cdn-avatars.huggingface.co/v1/production/uploads/631d760344503b7227837242/3b6JRusFX6GKJpsN9ZdeJ.png
Max Ku
vinesmsuic
user
vinesmsuic
6993ffc550fb2c0be4783dd1
Ka-Hei Hui
6993ffc550fb2c0be4783dd2
Ping Nie
6993ffc550fb2c0be4783dd3
Wenhu Chen
VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstruction
Evaluating whether Multimodal Large Language Models (MLLMs) genuinely reason about physical dynamics remains challenging. Most existing benchmarks rely on recognition-style protocols such as Visual Question Answering (VQA) and Violation of Expectation (VoE), which can often be answered without committing to an explicit, testable physical hypothesis. We propose VisPhyWorld, an execution-based framework that evaluates physical reasoning by requiring models to generate executable simulator code from visual observations. By producing runnable code, the inferred world representation is directly inspectable, editable, and falsifiable. This separates physical reasoning from rendering. Building on this framework, we introduce VisPhyBench, comprising 209 evaluation scenes derived from 108 physical templates and a systematic protocol that evaluates how well models reconstruct appearance and reproduce physically plausible motion. Our pipeline produces valid reconstructed videos in 97.7% on the benchmark. Experiments show that while state-of-the-art MLLMs achieve strong semantic scene understanding, they struggle to accurately infer physical parameters and to simulate consistent physical dynamics.
VisPhyWorld framework evaluates physical reasoning in MLLMs by requiring executable simulator code generation from visual observations, separating physical reasoning from rendering and enabling inspectable, falsifiable world representations.
Multimodal Large Language Models Visual Question Answering Violation of Expectation execution-based framework simulator code generation physical reasoning world representation physical parameters physical dynamics semantic scene understanding