GeoMotionGPT: Geometry-Aligned Motion Understanding with Large Language Models
Discrete motion tokenization has recently enabled Large Language Models (LLMs) to serve as versatile backbones for motion understanding and motion-language reasoning. However, existing pipelines typically decouple motion quantization from semantic embedding learning, linking them solely via token IDs. This approach fails to effectively align the intrinsic geometry of the motion space with the embedding space, thereby hindering the LLM's capacity for nuanced motion reasoning. We argue that alignment is most effective when both modalities share a unified geometric basis. Therefore, instead of forcing the LLM to reconstruct the complex geometry among motion tokens from scratch, we present a novel framework that explicitly enforces orthogonality on both the motion codebook and the LLM embedding space, ensuring that their relational structures naturally mirror each other. Specifically, we employ a decoder-only quantizer with Gumbel-Softmax for differentiable training and balanced codebook usage. To bridge the modalities, we use a sparse projection that maps motion codes into the LLM embedding space while preserving orthogonality. Finally, a two-stage orthonormal regularization schedule enforces soft constraints during tokenizer training and LLM fine-tuning to maintain geometric alignment without hindering semantic adaptation. Extensive experiments on HumanML3D demonstrate that our framework achieves a 20% performance improvement over current state-of-the-art methods, validating that a unified geometric basis effectively empowers the LLM for nuanced motion reasoning.
6966ab01c5e371f6b235cf6e
Zhankai Ye
claimed_verified
2026-01-14T09:52:38.382Z
66bb6f5a7686e47d84f111d1
/avatars/8f4ddca4ef47e07389a93ed126d8ffb6.svg
Zhankai Ye
zy22b
user
zy22b
6966ab01c5e371f6b235cf6f
Bofan Li
6966ab01c5e371f6b235cf70
Yukai Jin
6966ab01c5e371f6b235cf71
Shuoqiu Li
6966ab01c5e371f6b235cf72
Wei Wang
6966ab01c5e371f6b235cf73
Yanfu Zhang
6966ab01c5e371f6b235cf74
Shangqian Gao
claimed_verified
2026-02-09T21:07:18.161Z
6673b68e27be15fd5fa84aaf
/avatars/4584d95c05dd035b7184eec713a751bd.svg
Shangqian Gao
sgao84
user
sgao84
6966ab01c5e371f6b235cf75
Xin Liu
GeoMotionGPT: Geometry-Aligned Motion Understanding with Large Language Models
Discrete motion tokenization has recently enabled Large Language Models (LLMs) to serve as versatile backbones for motion understanding and motion-language reasoning. However, existing pipelines typically decouple motion quantization from semantic embedding learning, linking them solely via token IDs. This approach fails to effectively align the intrinsic geometry of the motion space with the embedding space, thereby hindering the LLM's capacity for nuanced motion reasoning. We argue that alignment is most effective when both modalities share a unified geometric basis. Therefore, instead of forcing the LLM to reconstruct the complex geometry among motion tokens from scratch, we present a novel framework that explicitly enforces orthogonality on both the motion codebook and the LLM embedding space, ensuring that their relational structures naturally mirror each other. Specifically, we employ a decoder-only quantizer with Gumbel-Softmax for differentiable training and balanced codebook usage. To bridge the modalities, we use a sparse projection that maps motion codes into the LLM embedding space while preserving orthogonality. Finally, a two-stage orthonormal regularization schedule enforces soft constraints during tokenizer training and LLM fine-tuning to maintain geometric alignment without hindering semantic adaptation. Extensive experiments on HumanML3D demonstrate that our framework achieves a 20% performance improvement over current state-of-the-art methods, validating that a unified geometric basis effectively empowers the LLM for nuanced motion reasoning.
A novel framework for motion understanding and motion-language reasoning that aligns motion token geometry with language model embeddings through orthogonal constraints and sparse projection techniques.
Large Language Models motion tokenization semantic embedding learning motion codebook Gumbel-Softmax sparse projection orthonormal regularization decoder-only quantizer motion-language reasoning geometric alignment