GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos
We present GenLCA, a diffusion-based generative model for generating and editing photorealistic full-body avatars from text and image inputs. The generated avatars are faithful to the inputs, while supporting high-fidelity facial and full-body animations. The core idea is a novel paradigm that enables training a full-body 3D diffusion model from partially observable 2D data, allowing the training dataset to scale to millions of real-world videos. This scalability contributes to the superior photorealism and generalizability of GenLCA. Specifically, we scale up the dataset by repurposing a pretrained feed-forward avatar reconstruction model as an animatable 3D tokenizer, which encodes unstructured video frames into structured 3D tokens. However, most real-world videos only provide partial observations of body parts, resulting in excessive blurring or transparency artifacts in the 3D tokens. To address this, we propose a novel visibility-aware diffusion training strategy that replaces invalid regions with learnable tokens and computes losses only over valid regions. We then train a flow-based diffusion model on the token dataset, inherently maintaining the photorealism and animatability provided by the pretrained avatar reconstruction model. Our approach effectively enables the use of large-scale real-world video data to train a diffusion model natively in 3D. We demonstrate the efficacy of our method through diverse and high-fidelity generation and editing results, outperforming existing solutions by a large margin. The project page is available at https://onethousandwu.com/GenLCA-Page.
69d739e8c6ed9968ac5c6a62
Yiqian Wu
claimed_verified
2026-04-09T08:06:22.683Z
65013802af1e4021439f8136
https://cdn-avatars.huggingface.co/v1/production/uploads/65013802af1e4021439f8136/7Yk_QElpJ5xlzqSkCuHvY.png
Yiqian Wu
onethousand
user
onethousand
69d739e8c6ed9968ac5c6a63
Rawal Khirodkar
69d739e8c6ed9968ac5c6a64
Egor Zakharov
69d739e8c6ed9968ac5c6a65
Timur Bagautdinov
69d739e8c6ed9968ac5c6a66
Lei Xiao
69d739e8c6ed9968ac5c6a67
Zhaoen Su
69d739e8c6ed9968ac5c6a68
Shunsuke Saito
69d739e8c6ed9968ac5c6a69
Xiaogang Jin
69d739e8c6ed9968ac5c6a6a
Junxuan Li
GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos
We present GenLCA, a diffusion-based generative model for generating and editing photorealistic full-body avatars from text and image inputs. The generated avatars are faithful to the inputs, while supporting high-fidelity facial and full-body animations. The core idea is a novel paradigm that enables training a full-body 3D diffusion model from partially observable 2D data, allowing the training dataset to scale to millions of real-world videos. This scalability contributes to the superior photorealism and generalizability of GenLCA. Specifically, we scale up the dataset by repurposing a pretrained feed-forward avatar reconstruction model as an animatable 3D tokenizer, which encodes unstructured video frames into structured 3D tokens. However, most real-world videos only provide partial observations of body parts, resulting in excessive blurring or transparency artifacts in the 3D tokens. To address this, we propose a novel visibility-aware diffusion training strategy that replaces invalid regions with learnable tokens and computes losses only over valid regions. We then train a flow-based diffusion model on the token dataset, inherently maintaining the photorealism and animatability provided by the pretrained avatar reconstruction model. Our approach effectively enables the use of large-scale real-world video data to train a diffusion model natively in 3D. We demonstrate the efficacy of our method through diverse and high-fidelity generation and editing results, outperforming existing solutions by a large margin. The project page is available at https://onethousandwu.com/GenLCA-Page.
GenLCA generates photorealistic 3D avatars from text and images using a novel diffusion model trained on large-scale 2D video data with visibility-aware training and flow-based diffusion.
diffusion-based generative model photorealistic avatars 3D diffusion model 2D data 3D tokens 3D tokenizer visibility-aware diffusion training flow-based diffusion model avatar reconstruction model animatable 3D tokens