ArtLLM: Generating Articulated Assets via 3D LLM
Creating interactive digital environments for gaming, robotics, and simulation relies on articulated 3D objects whose functionality emerges from their part geometry and kinematic structure. However, existing approaches remain fundamentally limited: optimization-based reconstruction methods require slow, per-object joint fitting and typically handle only simple, single-joint objects, while retrieval-based methods assemble parts from a fixed library, leading to repetitive geometry and poor generalization. To address these challenges, we introduce ArtLLM, a novel framework for generating high-quality articulated assets directly from complete 3D meshes. At its core is a 3D multimodal large language model trained on a large-scale articulation dataset curated from both existing articulation datasets and procedurally generated objects. Unlike prior work, ArtLLM autoregressively predicts a variable number of parts and joints, inferring their kinematic structure in a unified manner from the object's point cloud. This articulation-aware layout then conditions a 3D generative model to synthesize high-fidelity part geometries. Experiments on the PartNet-Mobility dataset show that ArtLLM significantly outperforms state-of-the-art methods in both part layout accuracy and joint prediction, while generalizing robustly to real-world objects. Finally, we demonstrate its utility in constructing digital twins, highlighting its potential for scalable robot learning.
69a66c32663a122a7ce667da
Penghao Wang
claimed_verified
2026-03-03T10:38:12.267Z
6462120032083c28b408e754
/avatars/f02c6d23f03420b716d995c92f8e2213.svg
Penghao Wang
AuWang
user
AuWang
69a66c32663a122a7ce667db
Siyuan Xie
69a66c32663a122a7ce667dc
Hongyu Yan
69a66c32663a122a7ce667dd
Xianghui Yang
69a66c32663a122a7ce667de
Jingwei Huang
69a66c32663a122a7ce667df
Chunchao Guo
69a66c32663a122a7ce667e0
Jiayuan Gu
ArtLLM: Generating Articulated Assets via 3D LLM
Creating interactive digital environments for gaming, robotics, and simulation relies on articulated 3D objects whose functionality emerges from their part geometry and kinematic structure. However, existing approaches remain fundamentally limited: optimization-based reconstruction methods require slow, per-object joint fitting and typically handle only simple, single-joint objects, while retrieval-based methods assemble parts from a fixed library, leading to repetitive geometry and poor generalization. To address these challenges, we introduce ArtLLM, a novel framework for generating high-quality articulated assets directly from complete 3D meshes. At its core is a 3D multimodal large language model trained on a large-scale articulation dataset curated from both existing articulation datasets and procedurally generated objects. Unlike prior work, ArtLLM autoregressively predicts a variable number of parts and joints, inferring their kinematic structure in a unified manner from the object's point cloud. This articulation-aware layout then conditions a 3D generative model to synthesize high-fidelity part geometries. Experiments on the PartNet-Mobility dataset show that ArtLLM significantly outperforms state-of-the-art methods in both part layout accuracy and joint prediction, while generalizing robustly to real-world objects. Finally, we demonstrate its utility in constructing digital twins, highlighting its potential for scalable robot learning.
ArtLLM generates articulated 3D assets from meshes using a 3D multimodal large language model that predicts part layouts and joints while synthesizing high-fidelity geometries.
3D multimodal large language model articulation-aware layout point cloud 3D generative model PartNet-Mobility dataset autoregressive prediction kinematic structure