Stepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports
Sports have long attracted broad attention as they push the limits of human physical and cognitive capabilities. Amid growing interest in spatial intelligence for vision-language models (VLMs), sports provide a natural testbed for understanding high-intensity human motion and dynamic object interactions. To this end, we present CourtSI, the first large-scale spatial intelligence dataset tailored to sports scenarios. CourtSI contains over 1M QA pairs, organized under a holistic taxonomy that systematically covers spatial counting, distance measurement, localization, and relational reasoning, across representative net sports including badminton, tennis, and table tennis. Leveraging well-defined court geometry as metric anchors, we develop a semi-automatic data engine to reconstruct sports scenes, enabling scalable curation of CourtSI. In addition, we introduce CourtSI-Bench, a high-quality evaluation benchmark comprising 3,686 QA pairs with rigorous human verification. We evaluate 25 proprietary and open-source VLMs on CourtSI-Bench, revealing a remaining human-AI performance gap and limited generalization from existing spatial intelligence benchmarks. These findings indicate that sports scenarios expose limitations in spatial intelligence capabilities captured by existing benchmarks. Further, fine-tuning Qwen3-VL-8B on CourtSI improves accuracy on CourtSI-Bench by 23.5 percentage points. The adapted model also generalizes effectively to CourtSI-Ext, an evaluation set built on a similar but unseen sport, and demonstrates enhanced spatial-aware commentary generation. Together, these findings demonstrate that CourtSI provides a scalable pathway toward advancing spatial intelligence of VLMs in sports.
69b0dd1d1f7902d47f6dfa31
Yuchen Yang
claimed_verified
2026-03-11T08:28:35.581Z
670f667d7d1b0373c63f1620
/avatars/21297c0b00cad7ea37c9167d113e2586.svg
Yuchen Yang
Charlie019
user
Charlie019
69b0dd1d1f7902d47f6dfa32
Yuqing Shao
69b0dd1d1f7902d47f6dfa33
Duxiu Huang
admin_assigned
2026-03-11T09:44:36.082Z
67ed714554dccfe33dec3a73
/avatars/3a249710fda58184159c8af3b5109273.svg
Duxiu Huang
Huangryyyy
user
Huangryyyy
69b0dd1d1f7902d47f6dfa34
Linfeng Dong
admin_assigned
2026-03-11T09:44:42.463Z
648180c78af4675862eff9f9
/avatars/bf8e688ebd67e6cb2719f4a52016765a.svg
Linfeng Dong
linfeng302
user
linfeng302
69b0dd1d1f7902d47f6dfa35
Yifei Liu
69b0dd1d1f7902d47f6dfa36
Suixin Tang
69b0dd1d1f7902d47f6dfa37
Xiang Zhou
69b0dd1d1f7902d47f6dfa38
Yuanyuan Gao
69b0dd1d1f7902d47f6dfa39
Wei Wang
69b0dd1d1f7902d47f6dfa3a
Yue Zhou
69b0dd1d1f7902d47f6dfa3b
Xue Yang
69b0dd1d1f7902d47f6dfa3c
Yanfeng Wang
69b0dd1d1f7902d47f6dfa3d
Xiao Sun
69b0dd1d1f7902d47f6dfa3e
Zhihang Zhong
claimed_verified
2026-03-11T08:28:37.615Z
6938f4de790b5cd0f6df6462
/avatars/4f22f0499d96bb749af7e8dba2b0b533.svg
Zhihang Zhong
Zuica96
user
Zuica96
Stepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports
Sports have long attracted broad attention as they push the limits of human physical and cognitive capabilities. Amid growing interest in spatial intelligence for vision-language models (VLMs), sports provide a natural testbed for understanding high-intensity human motion and dynamic object interactions. To this end, we present CourtSI, the first large-scale spatial intelligence dataset tailored to sports scenarios. CourtSI contains over 1M QA pairs, organized under a holistic taxonomy that systematically covers spatial counting, distance measurement, localization, and relational reasoning, across representative net sports including badminton, tennis, and table tennis. Leveraging well-defined court geometry as metric anchors, we develop a semi-automatic data engine to reconstruct sports scenes, enabling scalable curation of CourtSI. In addition, we introduce CourtSI-Bench, a high-quality evaluation benchmark comprising 3,686 QA pairs with rigorous human verification. We evaluate 25 proprietary and open-source VLMs on CourtSI-Bench, revealing a remaining human-AI performance gap and limited generalization from existing spatial intelligence benchmarks. These findings indicate that sports scenarios expose limitations in spatial intelligence capabilities captured by existing benchmarks. Further, fine-tuning Qwen3-VL-8B on CourtSI improves accuracy on CourtSI-Bench by 23.5 percentage points. The adapted model also generalizes effectively to CourtSI-Ext, an evaluation set built on a similar but unseen sport, and demonstrates enhanced spatial-aware commentary generation. Together, these findings demonstrate that CourtSI provides a scalable pathway toward advancing spatial intelligence of VLMs in sports.
CourtSI is a large-scale spatial intelligence dataset for sports scenarios that enables evaluation and improvement of vision-language models' understanding of human motion and object interactions.
spatial intelligence vision-language models CourtSI CourtSI-Bench CourtSI-Ext Qwen3-VL-8B fine-tuning spatial-aware commentary generation