PAI-Bench: A Comprehensive Benchmark For Physical AI
Physical AI aims to develop models that can perceive and predict real-world dynamics; yet, the extent to which current multi-modal large language models and video generative models support these abilities is insufficiently understood. We introduce Physical AI Bench (PAI-Bench), a unified and comprehensive benchmark that evaluates perception and prediction capabilities across video generation, conditional video generation, and video understanding, comprising 2,808 real-world cases with task-aligned metrics designed to capture physical plausibility and domain-specific reasoning. Our study provides a systematic assessment of recent models and shows that video generative models, despite strong visual fidelity, often struggle to maintain physically coherent dynamics, while multi-modal large language models exhibit limited performance in forecasting and causal interpretation. These observations suggest that current systems are still at an early stage in handling the perceptual and predictive demands of Physical AI. In summary, PAI-Bench establishes a realistic foundation for evaluating Physical AI and highlights key gaps that future systems must address.
692e9bc537312eaa83fd896b
Fengzhe Zhou
692e9bc537312eaa83fd896c
Jiannan Huang
claimed_verified
2025-12-03T09:17:39.299Z
63cfa23ceea5651bbb80d981
/avatars/2c6839fc71e91591bc26441b928d4dc5.svg
Jiannan Huang
Jiannan2003
user
Jiannan2003
692e9bc537312eaa83fd896d
Jialuo Li
claimed_verified
2025-12-05T08:30:11.570Z
6555a56f7cca2ba493f5331a
https://cdn-avatars.huggingface.co/v1/production/uploads/6555a56f7cca2ba493f5331a/d1_JZic4rFi80xe7m7VWs.jpeg
Jialuo Li
Jialuo21
user
Jialuo21
692e9bc537312eaa83fd896e
Deva Ramanan
692e9bc537312eaa83fd896f
Humphrey Shi
PAI-Bench: A Comprehensive Benchmark For Physical AI
Physical AI aims to develop models that can perceive and predict real-world dynamics; yet, the extent to which current multi-modal large language models and video generative models support these abilities is insufficiently understood. We introduce Physical AI Bench (PAI-Bench), a unified and comprehensive benchmark that evaluates perception and prediction capabilities across video generation, conditional video generation, and video understanding, comprising 2,808 real-world cases with task-aligned metrics designed to capture physical plausibility and domain-specific reasoning. Our study provides a systematic assessment of recent models and shows that video generative models, despite strong visual fidelity, often struggle to maintain physically coherent dynamics, while multi-modal large language models exhibit limited performance in forecasting and causal interpretation. These observations suggest that current systems are still at an early stage in handling the perceptual and predictive demands of Physical AI. In summary, PAI-Bench establishes a realistic foundation for evaluating Physical AI and highlights key gaps that future systems must address.
PAI-Bench evaluates the perception and prediction capabilities of multi-modal large language models and video generative models, revealing limitations in physical coherence and causal reasoning.
multi-modal large language models video generative models video generation conditional video generation video understanding physical plausibility domain-specific reasoning physically coherent dynamics forecasting causal interpretation