Pippo: High-Resolution Multi-View Humans from a Single Image
We present Pippo, a generative model capable of producing 1K resolution dense
turnaround videos of a person from a single casually clicked photo. Pippo is a
multi-view diffusion transformer and does not require any additional inputs -
e.g., a fitted parametric model or camera parameters of the input image. We
pre-train Pippo on 3B human images without captions, and conduct multi-view
mid-training and post-training on studio captured humans. During mid-training,
to quickly absorb the studio dataset, we denoise several (up to 48) views at
low-resolution, and encode target cameras coarsely using a shallow MLP. During
post-training, we denoise fewer views at high-resolution and use pixel-aligned
controls (e.g., Spatial anchor and Plucker rays) to enable 3D consistent
generations. At inference, we propose an attention biasing technique that
allows Pippo to simultaneously generate greater than 5 times as many views as
seen during training. Finally, we also introduce an improved metric to evaluate
3D consistency of multi-view generations, and show that Pippo outperforms
existing works on multi-view human generation from a single image.
67aceb3315317375eccddc3b
Yash Kant
claimed_verified
2025-02-13T08:24:09.195Z
638546d0a179f856005ae310
/avatars/f7f2fface336c8e168a1daaf9fd4d40c.svg
yashkant
yashkant
user
yashkant
67aceb3315317375eccddc3c
Ethan Weber
67aceb3315317375eccddc3d
Jin Kyu Kim
67aceb3315317375eccddc3e
Rawal Khirodkar
67aceb3315317375eccddc3f
Su Zhaoen
67aceb3315317375eccddc40
Julieta Martinez
67aceb3315317375eccddc41
Igor Gilitschenski
67aceb3315317375eccddc42
Shunsuke Saito
67aceb3315317375eccddc43
Timur Bagautdinov
Pippo: High-Resolution Multi-View Humans from a Single Image
We present Pippo, a generative model capable of producing 1K resolution dense
turnaround videos of a person from a single casually clicked photo. Pippo is a
multi-view diffusion transformer and does not require any additional inputs -
e.g., a fitted parametric model or camera parameters of the input image. We
pre-train Pippo on 3B human images without captions, and conduct multi-view
mid-training and post-training on studio captured humans. During mid-training,
to quickly absorb the studio dataset, we denoise several (up to 48) views at
low-resolution, and encode target cameras coarsely using a shallow MLP. During
post-training, we denoise fewer views at high-resolution and use pixel-aligned
controls (e.g., Spatial anchor and Plucker rays) to enable 3D consistent
generations. At inference, we propose an attention biasing technique that
allows Pippo to simultaneously generate greater than 5 times as many views as
seen during training. Finally, we also introduce an improved metric to evaluate
3D consistency of multi-view generations, and show that Pippo outperforms
existing works on multi-view human generation from a single image.
Pippo, a multi-view diffusion transformer model, generates high-resolution 3D consistent video turnarounds from a single photo using improved denoising techniques and attention biasing.
generative model diffusion transformer denoising multi-view camera parameters pre-training studio captured shallow MLP low-resolution high-resolution pixel-aligned controls Spatial anchor Plucker rays 3D consistent generations attention biasing 3D consistency metric