Stroke3D: Lifting 2D strokes into rigged 3D model via latent diffusion models
Rigged 3D assets are fundamental to 3D deformation and animation. However, existing 3D generation methods face challenges in generating animatable geometry, while rigging techniques lack fine-grained structural control over skeleton creation. To address these limitations, we introduce Stroke3D, a novel framework that directly generates rigged meshes from user inputs: 2D drawn strokes and a descriptive text prompt. Our approach pioneers a two-stage pipeline that separates the generation into: 1) Controllable Skeleton Generation, we employ the Skeletal Graph VAE (Sk-VAE) to encode the skeleton's graph structure into a latent space, where the Skeletal Graph DiT (Sk-DiT) generates a skeletal embedding. The generation process is conditioned on both the text for semantics and the 2D strokes for explicit structural control, with the VAE's decoder reconstructing the final high-quality 3D skeleton; and 2) Enhanced Mesh Synthesis via TextuRig and SKA-DPO, where we then synthesize a textured mesh conditioned on the generated skeleton. For this stage, we first enhance an existing skeleton-to-mesh model by augmenting its training data with TextuRig: a dataset of textured and rigged meshes with captions, curated from Objaverse-XL. Additionally, we employ a preference optimization strategy, SKA-DPO, guided by a skeleton-mesh alignment score, to further improve geometric fidelity. Together, our framework enables a more intuitive workflow for creating ready to animate 3D content. To the best of our knowledge, our work is the first to generate rigged 3D meshes conditioned on user-drawn 2D strokes. Extensive experiments demonstrate that Stroke3D produces plausible skeletons and high-quality meshes.
698beee66052d3bed9630a1c
Ruisi Zhao
claimed_verified
2026-02-11T11:17:15.023Z
641590e5486c7c9a5d13fe35
/avatars/2ef3432815e34a0eee45297fd99e5c40.svg
Ruisi Zhao
zhaors00
user
zhaors00
698beee66052d3bed9630a1d
Haoren Zheng
claimed_verified
2026-02-11T11:14:57.543Z
64173f238f689506e71091c4
/avatars/dbfe7939d3f2bb846df8447485295cdc.svg
Zheng Haoren
Zhroyn
user
Zhroyn
698beee66052d3bed9630a1e
Zongxin Yang
698beee66052d3bed9630a1f
Hehe Fan
698beee66052d3bed9630a20
Yi Yang
Stroke3D: Lifting 2D strokes into rigged 3D model via latent diffusion models
Rigged 3D assets are fundamental to 3D deformation and animation. However, existing 3D generation methods face challenges in generating animatable geometry, while rigging techniques lack fine-grained structural control over skeleton creation. To address these limitations, we introduce Stroke3D, a novel framework that directly generates rigged meshes from user inputs: 2D drawn strokes and a descriptive text prompt. Our approach pioneers a two-stage pipeline that separates the generation into: 1) Controllable Skeleton Generation, we employ the Skeletal Graph VAE (Sk-VAE) to encode the skeleton's graph structure into a latent space, where the Skeletal Graph DiT (Sk-DiT) generates a skeletal embedding. The generation process is conditioned on both the text for semantics and the 2D strokes for explicit structural control, with the VAE's decoder reconstructing the final high-quality 3D skeleton; and 2) Enhanced Mesh Synthesis via TextuRig and SKA-DPO, where we then synthesize a textured mesh conditioned on the generated skeleton. For this stage, we first enhance an existing skeleton-to-mesh model by augmenting its training data with TextuRig: a dataset of textured and rigged meshes with captions, curated from Objaverse-XL. Additionally, we employ a preference optimization strategy, SKA-DPO, guided by a skeleton-mesh alignment score, to further improve geometric fidelity. Together, our framework enables a more intuitive workflow for creating ready to animate 3D content. To the best of our knowledge, our work is the first to generate rigged 3D meshes conditioned on user-drawn 2D strokes. Extensive experiments demonstrate that Stroke3D produces plausible skeletons and high-quality meshes.
Stroke3D generates rigged 3D meshes from 2D strokes and text prompts through a two-stage pipeline combining controllable skeleton generation with enhanced mesh synthesis.
Skeletal Graph VAE Sk-VAE Skeletal Graph DiT Sk-DiT TextuRig SKA-DPO skeleton-mesh alignment score Objaverse-XL preference optimization controllable skeleton generation enhanced mesh synthesis