PhyX: Does Your Model Have the "Wits" for Physical Reasoning?
Existing benchmarks fail to capture a crucial aspect of intelligence:
physical reasoning, the integrated ability to combine domain knowledge,
symbolic reasoning, and understanding of real-world constraints. To address
this gap, we introduce PhyX: the first large-scale benchmark designed to assess
models capacity for physics-grounded reasoning in visual scenarios. PhyX
includes 3K meticulously curated multimodal questions spanning 6 reasoning
types across 25 sub-domains and 6 core physics domains: thermodynamics,
electromagnetism, mechanics, modern physics, optics, and wave\&acoustics. In
our comprehensive evaluation, even state-of-the-art models struggle
significantly with physical reasoning. GPT-4o, Claude3.7-Sonnet, and
GPT-o4-mini achieve only 32.5\%, 42.2\%, and 45.8\% accuracy
respectively-performance gaps exceeding 29\% compared to human experts. Our
analysis exposes critical limitations in current models: over-reliance on
memorized disciplinary knowledge, excessive dependence on mathematical
formulations, and surface-level visual pattern matching rather than genuine
physical understanding. We provide in-depth analysis through fine-grained
statistics, detailed case studies, and multiple evaluation paradigms to
thoroughly examine physical reasoning capabilities. To ensure reproducibility,
we implement a compatible evaluation protocol based on widely-used toolkits
such as VLMEvalKit, enabling one-click evaluation.
6830404effb59afb6569273a
Hui Shen
claimed_verified
2025-05-27T07:55:15.229Z
640feb91b0ee289c8583dd59
https://cdn-avatars.huggingface.co/v1/production/uploads/640feb91b0ee289c8583dd59/CA4UZj2mEsJO2UR0NnN5H.jpeg
Hui Shen
Cloudriver
user
Cloudriver
6830404effb59afb6569273b
Taiqiang Wu
claimed_verified
2025-05-26T08:14:30.851Z
6621cea88850e38ffbb1854f
https://cdn-avatars.huggingface.co/v1/production/uploads/6621cea88850e38ffbb1854f/qmKHts7YCUylxqzPHk4uT.jpeg
Taki WU
taki555
user
taki555
6830404effb59afb6569273c
Qi Han
claimed_verified
2025-05-27T07:55:12.818Z
6766d2764d57021c96679b7f
https://cdn-avatars.huggingface.co/v1/production/uploads/6766d2764d57021c96679b7f/_oJuienOnsMMPz_qZzVCb.png
QI HAN
HaibaraAI-APTX
user
HaibaraAI-APTX
6830404effb59afb6569273d
Yunta Hsieh
6830404effb59afb6569273e
Jizhou Wang
claimed_verified
2025-05-26T08:14:33.171Z
67fe265f9698ae4f5f4db718
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/0PLJEbUyJbM9BacFxcScP.png
Jizhou Wang
John-ai-bee
user
John-ai-bee
6830404effb59afb6569273f
Yuyue Zhang
6830404effb59afb65692740
Yuxin Cheng
6830404effb59afb65692741
Zijian Hao
6830404effb59afb65692742
Yuansheng Ni
claimed_verified
2025-05-28T09:01:10.443Z
64de37ee5e192985054be575
https://cdn-avatars.huggingface.co/v1/production/uploads/64de37ee5e192985054be575/fVV7JQMtp_J3uFqszJJHH.jpeg
Yuansheng Ni
yuanshengni
user
yuanshengni
6830404effb59afb65692743
Xin Wang
6830404effb59afb65692744
Zhongwei Wan
6830404effb59afb65692745
Kai Zhang
6830404effb59afb65692746
Wendong Xu
claimed_verified
2026-03-23T11:05:16.071Z
65437ef98f84f869869cb698
https://cdn-avatars.huggingface.co/v1/production/uploads/65437ef98f84f869869cb698/xRFcyXwQq7szB-GfAeoHu.png
Wendong XU
kirailol
user
kirailol
6830404effb59afb65692747
Jing Xiong
claimed_verified
2025-10-30T14:49:18.739Z
60851545a5da133ac6c38686
/avatars/d385fcc513acef80a3b711aa92d898e5.svg
Jing Xiong
menik1126
user
menik1126
6830404effb59afb65692748
Ping Luo
6830404effb59afb65692749
Wenhu Chen
6830404effb59afb6569274a
Chaofan Tao
claimed_verified
2025-11-24T08:00:54.310Z
60fc2fcca6bdebbe52dfdaf4
/avatars/1d59a7f33cb0df04678516f337e6b881.svg
Chaofan Tao
tcftrees
user
tcftrees
6830404effb59afb6569274b
Zhuoqing Mao
6830404effb59afb6569274c
Ngai Wong
PhyX: Does Your Model Have the "Wits" for Physical Reasoning?
Existing benchmarks fail to capture a crucial aspect of intelligence:
physical reasoning, the integrated ability to combine domain knowledge,
symbolic reasoning, and understanding of real-world constraints. To address
this gap, we introduce PhyX: the first large-scale benchmark designed to assess
models capacity for physics-grounded reasoning in visual scenarios. PhyX
includes 3K meticulously curated multimodal questions spanning 6 reasoning
types across 25 sub-domains and 6 core physics domains: thermodynamics,
electromagnetism, mechanics, modern physics, optics, and wave\&acoustics. In
our comprehensive evaluation, even state-of-the-art models struggle
significantly with physical reasoning. GPT-4o, Claude3.7-Sonnet, and
GPT-o4-mini achieve only 32.5\%, 42.2\%, and 45.8\% accuracy
respectively-performance gaps exceeding 29\% compared to human experts. Our
analysis exposes critical limitations in current models: over-reliance on
memorized disciplinary knowledge, excessive dependence on mathematical
formulations, and surface-level visual pattern matching rather than genuine
physical understanding. We provide in-depth analysis through fine-grained
statistics, detailed case studies, and multiple evaluation paradigms to
thoroughly examine physical reasoning capabilities. To ensure reproducibility,
we implement a compatible evaluation protocol based on widely-used toolkits
such as VLMEvalKit, enabling one-click evaluation.
A new benchmark, PhyX, evaluates models' physics-grounded reasoning in visual scenarios, revealing significant limitations in current models' physical understanding compared to human experts.
multimodal questions reasoning types sub-domains core physics domains thermodynamics electromagnetism mechanics modern physics optics wave\&acoustics fine-grained statistics case studies evaluation paradigms VLMEvalKit