STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D
  Intelligence
Despite rapid progress in Multi-modal Large Language Models and Large
Audio-Language Models, existing audio benchmarks largely test semantics that
can be recovered from text captions, masking deficits in fine-grained
perceptual reasoning. We formalize audio 4D intelligence that is defined as
reasoning over sound dynamics in time and 3D space, and introduce STAR-Bench to
measure it. STAR-Bench combines a Foundational Acoustic Perception setting (six
attributes under absolute and relative regimes) with a Holistic Spatio-Temporal
Reasoning setting that includes segment reordering for continuous and discrete
processes and spatial tasks spanning static localization, multi-source
relations, and dynamic trajectories. Our data curation pipeline uses two
methods to ensure high-quality samples. For foundational tasks, we use
procedurally synthesized and physics-simulated audio. For holistic data, we
follow a four-stage process that includes human annotation and final selection
based on human performance. Unlike prior benchmarks where caption-only
answering reduces accuracy slightly, STAR-Bench induces far larger drops
(-31.5\% temporal, -35.2\% spatial), evidencing its focus on linguistically
hard-to-describe cues. Evaluating 19 models reveals substantial gaps compared
with humans and a capability hierarchy: closed-source models are bottlenecked
by fine-grained perception, while open-source models lag across perception,
knowledge, and reasoning. Our STAR-Bench provides critical insights and a clear
path forward for developing future models with a more robust understanding of
the physical world.
69018af4646208eac0d1f4a8
Zihan Liu
69018af4646208eac0d1f4a9
Zhikang Niu
69018af4646208eac0d1f4aa
Qiuyang Xiao
69018af4646208eac0d1f4ab
Zhisheng Zheng
69018af4646208eac0d1f4ac
Ruoqi Yuan
69018af4646208eac0d1f4ad
Yuhang Zang
claimed_verified
2025-10-29T12:41:27.000Z
63859cf3b2906edaf83af9f0
https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/kajwuVzd4pDucSPlwghxo.png
Yuhang Zang
yuhangzang
user
yuhangzang
69018af4646208eac0d1f4ae
Yuhang Cao
69018af4646208eac0d1f4af
Xiaoyi Dong
69018af4646208eac0d1f4b0
Jianze Liang
69018af4646208eac0d1f4b1
Xie Chen
69018af4646208eac0d1f4b2
Leilei Sun
69018af4646208eac0d1f4b3
Dahua Lin
69018af4646208eac0d1f4b4
Jiaqi Wang
claimed_verified
2025-10-29T12:41:23.657Z
64b4eec4faa3181a5eab9c46
/avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
Jiaqi Wang
myownskyW7
user
myownskyW7
STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D
  Intelligence
Despite rapid progress in Multi-modal Large Language Models and Large
Audio-Language Models, existing audio benchmarks largely test semantics that
can be recovered from text captions, masking deficits in fine-grained
perceptual reasoning. We formalize audio 4D intelligence that is defined as
reasoning over sound dynamics in time and 3D space, and introduce STAR-Bench to
measure it. STAR-Bench combines a Foundational Acoustic Perception setting (six
attributes under absolute and relative regimes) with a Holistic Spatio-Temporal
Reasoning setting that includes segment reordering for continuous and discrete
processes and spatial tasks spanning static localization, multi-source
relations, and dynamic trajectories. Our data curation pipeline uses two
methods to ensure high-quality samples. For foundational tasks, we use
procedurally synthesized and physics-simulated audio. For holistic data, we
follow a four-stage process that includes human annotation and final selection
based on human performance. Unlike prior benchmarks where caption-only
answering reduces accuracy slightly, STAR-Bench induces far larger drops
(-31.5\% temporal, -35.2\% spatial), evidencing its focus on linguistically
hard-to-describe cues. Evaluating 19 models reveals substantial gaps compared
with humans and a capability hierarchy: closed-source models are bottlenecked
by fine-grained perception, while open-source models lag across perception,
knowledge, and reasoning. Our STAR-Bench provides critical insights and a clear
path forward for developing future models with a more robust understanding of
the physical world.
STAR-Bench measures audio 4D intelligence by evaluating sound dynamics in time and 3D space, revealing gaps in fine-grained perceptual reasoning among existing models.
Multi-modal Large Language Models Large Audio-Language Models audio benchmarks sound dynamics 3D space STAR-Bench Foundational Acoustic Perception Holistic Spatio-Temporal Reasoning segment reordering static localization multi-source relations dynamic trajectories procedurally synthesized physics-simulated audio human annotation human performance closed-source models open-source models fine-grained perception knowledge reasoning