The Quest for Generalizable Motion Generation: Data, Model, and
  Evaluation
Despite recent advances in 3D human motion generation (MoGen) on standard
benchmarks, existing models still face a fundamental bottleneck in their
generalization capability. In contrast, adjacent generative fields, most
notably video generation (ViGen), have demonstrated remarkable generalization
in modeling human behaviors, highlighting transferable insights that MoGen can
leverage. Motivated by this observation, we present a comprehensive framework
that systematically transfers knowledge from ViGen to MoGen across three key
pillars: data, modeling, and evaluation. First, we introduce ViMoGen-228K, a
large-scale dataset comprising 228,000 high-quality motion samples that
integrates high-fidelity optical MoCap data with semantically annotated motions
from web videos and synthesized samples generated by state-of-the-art ViGen
models. The dataset includes both text-motion pairs and text-video-motion
triplets, substantially expanding semantic diversity. Second, we propose
ViMoGen, a flow-matching-based diffusion transformer that unifies priors from
MoCap data and ViGen models through gated multimodal conditioning. To enhance
efficiency, we further develop ViMoGen-light, a distilled variant that
eliminates video generation dependencies while preserving strong
generalization. Finally, we present MBench, a hierarchical benchmark designed
for fine-grained evaluation across motion quality, prompt fidelity, and
generalization ability. Extensive experiments show that our framework
significantly outperforms existing approaches in both automatic and human
evaluations. The code, data, and benchmark will be made publicly available.
690422312c556835fa67f096
Jing Lin
690422312c556835fa67f097
Ruisi Wang
claimed_verified
2026-02-24T09:52:17.756Z
65b74305e602b6c2c9125480
https://cdn-avatars.huggingface.co/v1/production/uploads/65b74305e602b6c2c9125480/mooL0oaSJX4b3aVGH7JZr.jpeg
wang ruisi
wruisi
user
wruisi
690422312c556835fa67f098
Junzhe Lu
690422312c556835fa67f099
Ziqi Huang
claimed_verified
2025-10-31T14:26:59.878Z
60efe7fa0d920bc7805cada5
https://cdn-avatars.huggingface.co/v1/production/uploads/60efe7fa0d920bc7805cada5/2LBrJBjSCOP5ilZIpWLHl.png
Ziqi Huang
Ziqi
user
Ziqi
690422312c556835fa67f09a
Guorui Song
690422312c556835fa67f09b
Ailing Zeng
690422312c556835fa67f09c
Xian Liu
690422312c556835fa67f09d
Chen Wei
690422312c556835fa67f09e
Wanqi Yin
claimed_verified
2026-03-23T11:03:50.900Z
668f51fb8e8d87dbdd23caa9
/avatars/0b17cb0f3ad2c729f185cdccdad94e48.svg
Yin Wanqi
waanqii
user
waanqii
690422312c556835fa67f09f
Qingping Sun
690422312c556835fa67f0a0
Zhongang Cai
claimed_verified
2026-02-24T09:52:19.435Z
652d06833b5997ed71ce5c46
https://cdn-avatars.huggingface.co/v1/production/uploads/652d06833b5997ed71ce5c46/O_D6bpa5mGxLA7uCjmVCG.jpeg
Zhongang Cai
caizhongang
user
caizhongang
690422312c556835fa67f0a1
Lei Yang
claimed_verified
2026-02-25T17:33:56.538Z
6626a471430a124253f197c8
https://cdn-avatars.huggingface.co/v1/production/uploads/6626a471430a124253f197c8/uVEk5nnW-bS6-no0rQ7Wh.png
yl-1993
yl-1993
user
yl-1993
690422312c556835fa67f0a2
Ziwei Liu
The Quest for Generalizable Motion Generation: Data, Model, and
  Evaluation
Despite recent advances in 3D human motion generation (MoGen) on standard
benchmarks, existing models still face a fundamental bottleneck in their
generalization capability. In contrast, adjacent generative fields, most
notably video generation (ViGen), have demonstrated remarkable generalization
in modeling human behaviors, highlighting transferable insights that MoGen can
leverage. Motivated by this observation, we present a comprehensive framework
that systematically transfers knowledge from ViGen to MoGen across three key
pillars: data, modeling, and evaluation. First, we introduce ViMoGen-228K, a
large-scale dataset comprising 228,000 high-quality motion samples that
integrates high-fidelity optical MoCap data with semantically annotated motions
from web videos and synthesized samples generated by state-of-the-art ViGen
models. The dataset includes both text-motion pairs and text-video-motion
triplets, substantially expanding semantic diversity. Second, we propose
ViMoGen, a flow-matching-based diffusion transformer that unifies priors from
MoCap data and ViGen models through gated multimodal conditioning. To enhance
efficiency, we further develop ViMoGen-light, a distilled variant that
eliminates video generation dependencies while preserving strong
generalization. Finally, we present MBench, a hierarchical benchmark designed
for fine-grained evaluation across motion quality, prompt fidelity, and
generalization ability. Extensive experiments show that our framework
significantly outperforms existing approaches in both automatic and human
evaluations. The code, data, and benchmark will be made publicly available.
A framework transfers knowledge from video generation to 3D human motion generation using a large dataset, a flow-matching-based diffusion transformer, and a hierarchical benchmark, improving generalization and performance.
3D human motion generation video generation ViMoGen-228K flow-matching-based diffusion transformer gated multimodal conditioning ViMoGen-light MBench hierarchical benchmark