Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizer
Prior motion generation largely follows two paradigms: continuous diffusion models that excel at kinematic control, and discrete token-based generators that are effective for semantic conditioning. To combine their strengths, we propose a three-stage framework comprising condition feature extraction (Perception), discrete token generation (Planning), and diffusion-based motion synthesis (Control). Central to this framework is MoTok, a diffusion-based discrete motion tokenizer that decouples semantic abstraction from fine-grained reconstruction by delegating motion recovery to a diffusion decoder, enabling compact single-layer tokens while preserving motion fidelity. For kinematic conditions, coarse constraints guide token generation during planning, while fine-grained constraints are enforced during control through diffusion-based optimization. This design prevents kinematic details from disrupting semantic token planning. On HumanML3D, our method significantly improves controllability and fidelity over MaskControl while using only one-sixth of the tokens, reducing trajectory error from 0.72 cm to 0.08 cm and FID from 0.083 to 0.029. Unlike prior methods that degrade under stronger kinematic constraints, ours improves fidelity, reducing FID from 0.033 to 0.014.
69bcb3c1081b9ad49af57259
Chenyang Gu
claimed_verified
2026-03-23T10:54:01.989Z
690429d49d820cebdc618411
/avatars/74504e5c8de5be64d4791143ff46bef5.svg
Gu Chenyang
rheallyc
user
rheallyc
69bcb3c1081b9ad49af5725a
Mingyuan Zhang
69bcb3c1081b9ad49af5725b
Haozhe Xie
claimed_verified
2026-04-01T08:36:44.238Z
63f47b5321eb234ab739e91a
https://cdn-avatars.huggingface.co/v1/production/uploads/63f47b5321eb234ab739e91a/vWfFNVtMkHl8gieha5PPd.jpeg
Haozhe Xie
hzxie
user
hzxie
69bcb3c1081b9ad49af5725c
Zhongang Cai
claimed_verified
2026-03-23T10:54:00.012Z
652d06833b5997ed71ce5c46
https://cdn-avatars.huggingface.co/v1/production/uploads/652d06833b5997ed71ce5c46/O_D6bpa5mGxLA7uCjmVCG.jpeg
Zhongang Cai
caizhongang
user
caizhongang
69bcb3c1081b9ad49af5725d
Lei Yang
claimed_verified
2026-03-23T10:53:29.079Z
6626a471430a124253f197c8
https://cdn-avatars.huggingface.co/v1/production/uploads/6626a471430a124253f197c8/uVEk5nnW-bS6-no0rQ7Wh.png
yl-1993
yl-1993
user
yl-1993
69bcb3c1081b9ad49af5725e
Ziwei Liu
Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizer
Prior motion generation largely follows two paradigms: continuous diffusion models that excel at kinematic control, and discrete token-based generators that are effective for semantic conditioning. To combine their strengths, we propose a three-stage framework comprising condition feature extraction (Perception), discrete token generation (Planning), and diffusion-based motion synthesis (Control). Central to this framework is MoTok, a diffusion-based discrete motion tokenizer that decouples semantic abstraction from fine-grained reconstruction by delegating motion recovery to a diffusion decoder, enabling compact single-layer tokens while preserving motion fidelity. For kinematic conditions, coarse constraints guide token generation during planning, while fine-grained constraints are enforced during control through diffusion-based optimization. This design prevents kinematic details from disrupting semantic token planning. On HumanML3D, our method significantly improves controllability and fidelity over MaskControl while using only one-sixth of the tokens, reducing trajectory error from 0.72 cm to 0.08 cm and FID from 0.083 to 0.029. Unlike prior methods that degrade under stronger kinematic constraints, ours improves fidelity, reducing FID from 0.033 to 0.014.
A three-stage motion generation framework combines discrete token-based planning with diffusion-based synthesis to improve controllability and fidelity while reducing token usage and computational requirements.
diffusion models discrete token-based generators motion synthesis motion tokenizer diffusion decoder kinematic control semantic conditioning token generation trajectory error FID