Dense Motion Captioning
Recent advances in 3D human motion and language integration have primarily
focused on text-to-motion generation, leaving the task of motion understanding
relatively unexplored. We introduce Dense Motion Captioning, a novel task that
aims to temporally localize and caption actions within 3D human motion
sequences. Current datasets fall short in providing detailed temporal
annotations and predominantly consist of short sequences featuring few actions.
To overcome these limitations, we present the Complex Motion Dataset (CompMo),
the first large-scale dataset featuring richly annotated, complex motion
sequences with precise temporal boundaries. Built through a carefully designed
data generation pipeline, CompMo includes 60,000 motion sequences, each
composed of multiple actions ranging from at least two to ten, accurately
annotated with their temporal extents. We further present DEMO, a model that
integrates a large language model with a simple motion adapter, trained to
generate dense, temporally grounded captions. Our experiments show that DEMO
substantially outperforms existing methods on CompMo as well as on adapted
benchmarks, establishing a robust baseline for future research in 3D motion
understanding and captioning.
69114e51830c57fd4c795fab
Shiyao Xu
claimed_verified
2025-11-10T21:03:08.576Z
6457920bcd935d48a4789cef
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/AswyuYQah0noLTYkpAmRT.png
Shiyao Xu
Xusy2333
user
Xusy2333
69114e51830c57fd4c795fac
Benedetta Liberatori
69114e51830c57fd4c795fad
Gül Varol
69114e51830c57fd4c795fae
Paolo Rota
Dense Motion Captioning
Recent advances in 3D human motion and language integration have primarily
focused on text-to-motion generation, leaving the task of motion understanding
relatively unexplored. We introduce Dense Motion Captioning, a novel task that
aims to temporally localize and caption actions within 3D human motion
sequences. Current datasets fall short in providing detailed temporal
annotations and predominantly consist of short sequences featuring few actions.
To overcome these limitations, we present the Complex Motion Dataset (CompMo),
the first large-scale dataset featuring richly annotated, complex motion
sequences with precise temporal boundaries. Built through a carefully designed
data generation pipeline, CompMo includes 60,000 motion sequences, each
composed of multiple actions ranging from at least two to ten, accurately
annotated with their temporal extents. We further present DEMO, a model that
integrates a large language model with a simple motion adapter, trained to
generate dense, temporally grounded captions. Our experiments show that DEMO
substantially outperforms existing methods on CompMo as well as on adapted
benchmarks, establishing a robust baseline for future research in 3D motion
understanding and captioning.
A new task, Dense Motion Captioning, is introduced with a large-scale dataset, CompMo, and a model, DEMO, that integrates a language model with a motion adapter to generate detailed, temporally grounded captions for 3D human motion sequences.
Dense Motion Captioning Complex Motion Dataset CompMo DEMO large language model motion adapter temporal localization temporal boundaries motion sequences action captioning