DiP: Taming Diffusion Models in Pixel Space
Diffusion models face a fundamental trade-off between generation quality and computational efficiency. Latent Diffusion Models (LDMs) offer an efficient solution but suffer from potential information loss and non-end-to-end training. In contrast, existing pixel space models bypass VAEs but are computationally prohibitive for high-resolution synthesis. To resolve this dilemma, we propose DiP, an efficient pixel space diffusion framework. DiP decouples generation into a global and a local stage: a Diffusion Transformer (DiT) backbone operates on large patches for efficient global structure construction, while a co-trained lightweight Patch Detailer Head leverages contextual features to restore fine-grained local details. This synergistic design achieves computational efficiency comparable to LDMs without relying on a VAE. DiP is accomplished with up to 10times faster inference speeds than previous method while increasing the total number of parameters by only 0.3%, and achieves an 1.79 FID score on ImageNet 256times256.
692d00ed4397b1ec214f677d
Zhennan Chen
admin_assigned
2025-12-01T10:15:12.572Z
66449e619ff401732687f013
/avatars/251897d1324a70a9bf761513871c5841.svg
chen
zhen-nan
user
zhen-nan
692d00ed4397b1ec214f677e
Junwei Zhu
admin_assigned
2025-12-01T10:15:20.475Z
65aadb2b9aba49e1d03cfccc
/avatars/b7c48dc999bf2f94018315b546d50eeb.svg
Junwei Zhu
ktpt123
user
ktpt123
692d00ed4397b1ec214f677f
Xu Chen
692d00ed4397b1ec214f6780
Jiangning Zhang
692d00ed4397b1ec214f6781
Xiaobin Hu
692d00ed4397b1ec214f6782
Hanzhen Zhao
692d00ed4397b1ec214f6783
Chengjie Wang
admin_assigned
2025-12-01T10:15:53.408Z
65729087379284b904a6d81d
/avatars/0e7968b1d9221d523a54e3ac787e449d.svg
Chengjie Wang
chengjie-wang
user
chengjie-wang
692d00ed4397b1ec214f6784
Jian Yang
692d00ed4397b1ec214f6785
Ying Tai
claimed_verified
2025-12-01T09:11:20.615Z
65734004769f3ee9bde1af10
/avatars/d6310ed861972fd691687d8f47413f33.svg
Ying Tai
yingtai
user
yingtai
DiP: Taming Diffusion Models in Pixel Space
Diffusion models face a fundamental trade-off between generation quality and computational efficiency. Latent Diffusion Models (LDMs) offer an efficient solution but suffer from potential information loss and non-end-to-end training. In contrast, existing pixel space models bypass VAEs but are computationally prohibitive for high-resolution synthesis. To resolve this dilemma, we propose DiP, an efficient pixel space diffusion framework. DiP decouples generation into a global and a local stage: a Diffusion Transformer (DiT) backbone operates on large patches for efficient global structure construction, while a co-trained lightweight Patch Detailer Head leverages contextual features to restore fine-grained local details. This synergistic design achieves computational efficiency comparable to LDMs without relying on a VAE. DiP is accomplished with up to 10times faster inference speeds than previous method while increasing the total number of parameters by only 0.3%, and achieves an 1.79 FID score on ImageNet 256times256.
DiP, a pixel space diffusion framework, combines a Diffusion Transformer and a Patch Detailer Head to achieve computational efficiency and high-quality image generation without using VAEs.
diffusion models latent diffusion models (LDMs) pixel space models variational autoencoders (VAEs) diffusion transformer (DiT) patch detailer head FID score