FrankenMotion: Part-level Human Motion Generation and Composition
Human motion generation from text prompts has made remarkable progress in recent years. However, existing methods primarily rely on either sequence-level or action-level descriptions due to the absence of fine-grained, part-level motion annotations. This limits their controllability over individual body parts. In this work, we construct a high-quality motion dataset with atomic, temporally-aware part-level text annotations, leveraging the reasoning capabilities of large language models (LLMs). Unlike prior datasets that either provide synchronized part captions with fixed time segments or rely solely on global sequence labels, our dataset captures asynchronous and semantically distinct part movements at fine temporal resolution. Based on this dataset, we introduce a diffusion-based part-aware motion generation framework, namely FrankenMotion, where each body part is guided by its own temporally-structured textual prompt. This is, to our knowledge, the first work to provide atomic, temporally-aware part-level motion annotations and have a model that allows motion generation with both spatial (body part) and temporal (atomic action) control. Experiments demonstrate that FrankenMotion outperforms all previous baseline models adapted and retrained for our setting, and our model can compose motions unseen during training. Our code and dataset will be publicly available upon publication.
696d9afc3f1837bfb897099b
Chuqiao Li
admin_assigned
2026-01-19T14:42:30.132Z
673c86d3cfd6e90464a294e2
https://cdn-avatars.huggingface.co/v1/production/uploads/673c86d3cfd6e90464a294e2/_wbiQRqNw1VD2mutOKrrm.jpeg
Chuqiao Li
Coral79
user
Coral79
696d9afc3f1837bfb897099c
Xianghui Xie
admin_assigned
2026-01-19T14:42:16.683Z
6966b9cff869182443fb3fde
/avatars/925b80085ded250c6bd8d412f2f4b636.svg
Xianghui Xie
xianghuix
user
xianghuix
696d9afc3f1837bfb897099d
Yong Cao
admin_assigned
2026-01-19T14:42:10.525Z
6170f862544a1c62b298b891
https://cdn-avatars.huggingface.co/v1/production/uploads/1634793843654-6170f862544a1c62b298b891.jpeg
Yong Cao
Yongcao
user
Yongcao
696d9afc3f1837bfb897099e
Andreas Geiger
admin_assigned
2026-01-19T14:42:04.135Z
620cae049086f3c07f01e3d5
https://cdn-avatars.huggingface.co/v1/production/uploads/1644998139538-noauth.jpeg
Andreas Geiger
andreas-geiger
user
andreas-geiger
696d9afc3f1837bfb897099f
Gerard Pons-Moll
FrankenMotion: Part-level Human Motion Generation and Composition
Human motion generation from text prompts has made remarkable progress in recent years. However, existing methods primarily rely on either sequence-level or action-level descriptions due to the absence of fine-grained, part-level motion annotations. This limits their controllability over individual body parts. In this work, we construct a high-quality motion dataset with atomic, temporally-aware part-level text annotations, leveraging the reasoning capabilities of large language models (LLMs). Unlike prior datasets that either provide synchronized part captions with fixed time segments or rely solely on global sequence labels, our dataset captures asynchronous and semantically distinct part movements at fine temporal resolution. Based on this dataset, we introduce a diffusion-based part-aware motion generation framework, namely FrankenMotion, where each body part is guided by its own temporally-structured textual prompt. This is, to our knowledge, the first work to provide atomic, temporally-aware part-level motion annotations and have a model that allows motion generation with both spatial (body part) and temporal (atomic action) control. Experiments demonstrate that FrankenMotion outperforms all previous baseline models adapted and retrained for our setting, and our model can compose motions unseen during training. Our code and dataset will be publicly available upon publication.
A diffusion-based framework generates human motion from text prompts with fine-grained part-level control using a newly constructed dataset with atomic, temporally-aware annotations.
diffusion-based part-aware motion generation large language models temporally-aware part-level text annotations atomic motion annotations temporally-structured textual prompts motion generation