DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformers
Diffusion Transformers (DiTs) have achieved state-of-the-art performance in image and video generation, but their success comes at the cost of heavy computation. This inefficiency is largely due to the fixed tokenization process, which uses constant-sized patches throughout the entire denoising phase, regardless of the content's complexity. We propose dynamic tokenization, an efficient test-time strategy that varies patch sizes based on content complexity and the denoising timestep. Our key insight is that early timesteps only require coarser patches to model global structure, while later iterations demand finer (smaller-sized) patches to refine local details. During inference, our method dynamically reallocates patch sizes across denoising steps for image and video generation and substantially reduces cost while preserving perceptual generation quality. Extensive experiments demonstrate the effectiveness of our approach: it achieves up to 3.52times and 3.2times speedup on FLUX-1.Dev and Wan 2.1, respectively, without compromising the generation quality and prompt adherence.
6997ceac7a658569d5a1016b
Dahye Kim
claimed_verified
2026-02-20T08:36:23.660Z
64bcc06fb567ae97c3272d3d
/avatars/bcb61fe9e575154d84913a1501971f1a.svg
kim
dahyekim
user
dahyekim
6997ceac7a658569d5a1016c
Deepti Ghadiyaram
claimed_verified
2026-04-13T07:38:41.801Z
6905068e4c9138632af2fa21
/avatars/c32215347601de938c218e6c81dc2e33.svg
Deepti Ghadiyaram
dghadiya
user
dghadiya
6997ceac7a658569d5a1016d
Raghudeep Gadde
admin_assigned
2026-02-20T09:04:41.675Z
6621eba3d93296af86f59d43
/avatars/1ee9bbaab46ba6eb237fe797d03a1dbd.svg
Raghudeep Gadde
raghudeepg
user
raghudeepg
DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformers
Diffusion Transformers (DiTs) have achieved state-of-the-art performance in image and video generation, but their success comes at the cost of heavy computation. This inefficiency is largely due to the fixed tokenization process, which uses constant-sized patches throughout the entire denoising phase, regardless of the content's complexity. We propose dynamic tokenization, an efficient test-time strategy that varies patch sizes based on content complexity and the denoising timestep. Our key insight is that early timesteps only require coarser patches to model global structure, while later iterations demand finer (smaller-sized) patches to refine local details. During inference, our method dynamically reallocates patch sizes across denoising steps for image and video generation and substantially reduces cost while preserving perceptual generation quality. Extensive experiments demonstrate the effectiveness of our approach: it achieves up to 3.52times and 3.2times speedup on FLUX-1.Dev and Wan 2.1, respectively, without compromising the generation quality and prompt adherence.
Dynamic tokenization improves diffusion transformer efficiency by adjusting patch sizes based on content complexity and denoising timestep, achieving significant speedup without quality loss.
diffusion transformers tokenization denoising phase patch sizes content complexity denoising timestep inference computational efficiency perceptual quality speedup