PixelDiT: Pixel Diffusion Transformers for Image Generation
Latent-space modeling has been the standard for Diffusion Transformers (DiTs). However, it relies on a two-stage pipeline where the pretrained autoencoder introduces lossy reconstruction, leading to error accumulation while hindering joint optimization. To address these issues, we propose PixelDiT, a single-stage, end-to-end model that eliminates the need for the autoencoder and learns the diffusion process directly in the pixel space. PixelDiT adopts a fully transformer-based architecture shaped by a dual-level design: a patch-level DiT that captures global semantics and a pixel-level DiT that refines texture details, enabling efficient training of a pixel-space diffusion model while preserving fine details. Our analysis reveals that effective pixel-level token modeling is essential to the success of pixel diffusion. PixelDiT achieves 1.61 FID on ImageNet 256x256, surpassing existing pixel generative models by a large margin. We further extend PixelDiT to text-to-image generation and pretrain it at the 1024x1024 resolution in pixel space. It achieves 0.74 on GenEval and 83.5 on DPG-bench, approaching the best latent diffusion models.
692ffb0d26742347f61daee9
Yongsheng Yu
claimed_verified
2025-12-04T08:49:00.220Z
635b6bdfd63bcb528acefa78
/avatars/54c62b41c147d0450bdfc72121860bb4.svg
Yongsheng Yu
yeates
user
yeates
692ffb0d26742347f61daeea
Wei Xiong
692ffb0d26742347f61daeeb
Weili Nie
692ffb0d26742347f61daeec
Yichen Sheng
692ffb0d26742347f61daeed
Shiqiu Liu
692ffb0d26742347f61daeee
Jiebo Luo
PixelDiT: Pixel Diffusion Transformers for Image Generation
Latent-space modeling has been the standard for Diffusion Transformers (DiTs). However, it relies on a two-stage pipeline where the pretrained autoencoder introduces lossy reconstruction, leading to error accumulation while hindering joint optimization. To address these issues, we propose PixelDiT, a single-stage, end-to-end model that eliminates the need for the autoencoder and learns the diffusion process directly in the pixel space. PixelDiT adopts a fully transformer-based architecture shaped by a dual-level design: a patch-level DiT that captures global semantics and a pixel-level DiT that refines texture details, enabling efficient training of a pixel-space diffusion model while preserving fine details. Our analysis reveals that effective pixel-level token modeling is essential to the success of pixel diffusion. PixelDiT achieves 1.61 FID on ImageNet 256x256, surpassing existing pixel generative models by a large margin. We further extend PixelDiT to text-to-image generation and pretrain it at the 1024x1024 resolution in pixel space. It achieves 0.74 on GenEval and 83.5 on DPG-bench, approaching the best latent diffusion models.
PixelDiT is a single-stage, end-to-end diffusion model that operates directly in pixel space, overcoming the limitations of latent-space modeling by using a dual-level transformer architecture and achieving competitive performance in image and text-to-image generation.
Latent-space modeling Diffusion Transformers DiTs PixelDiT pretrained autoencoder pixel space fully transformer-based architecture patch-level DiT pixel-level DiT token modeling FID ImageNet GenEval DPG-bench