One Model, Many Budgets: Elastic Latent Interfaces for Diffusion Transformers
Diffusion transformers (DiTs) achieve high generative quality but lock FLOPs to image resolution, limiting principled latency-quality trade-offs, and allocate computation uniformly across input spatial tokens, wasting resource allocation to unimportant regions. We introduce Elastic Latent Interface Transformer (ELIT), a drop-in, DiT-compatible mechanism that decouples input image size from compute. Our approach inserts a latent interface, a learnable variable-length token sequence on which standard transformer blocks can operate. Lightweight Read and Write cross-attention layers move information between spatial tokens and latents and prioritize important input regions. By training with random dropping of tail latents, ELIT learns to produce importance-ordered representations with earlier latents capturing global structure while later ones contain information to refine details. At inference, the number of latents can be dynamically adjusted to match compute constraints. ELIT is deliberately minimal, adding two cross-attention layers while leaving the rectified flow objective and the DiT stack unchanged. Across datasets and architectures (DiT, U-ViT, HDiT, MM-DiT), ELIT delivers consistent gains. On ImageNet-1K 512px, ELIT delivers an average gain of 35.3% and 39.6% in FID and FDD scores. Project page: https://snap-research.github.io/elit/
69b3746c2ad6c5d4cf486a11
Moayed Haji-Ali
69b3746c2ad6c5d4cf486a12
Willi Menapace
69b3746c2ad6c5d4cf486a13
Ivan Skorokhodov
69b3746c2ad6c5d4cf486a14
Dogyun Park
69b3746c2ad6c5d4cf486a15
Anil Kag
claimed_verified
2026-03-17T08:21:14.569Z
69b84d69b6bcc376261b1b55
/avatars/87692477b406c4fc7f689316b81d50a7.svg
Anil Kag
akag-snap
user
akag-snap
69b3746c2ad6c5d4cf486a16
Michael Vasilkovsky
69b3746c2ad6c5d4cf486a17
Sergey Tulyakov
69b3746c2ad6c5d4cf486a18
Vicente Ordonez
69b3746c2ad6c5d4cf486a19
Aliaksandr Siarohin
One Model, Many Budgets: Elastic Latent Interfaces for Diffusion Transformers
Diffusion transformers (DiTs) achieve high generative quality but lock FLOPs to image resolution, limiting principled latency-quality trade-offs, and allocate computation uniformly across input spatial tokens, wasting resource allocation to unimportant regions. We introduce Elastic Latent Interface Transformer (ELIT), a drop-in, DiT-compatible mechanism that decouples input image size from compute. Our approach inserts a latent interface, a learnable variable-length token sequence on which standard transformer blocks can operate. Lightweight Read and Write cross-attention layers move information between spatial tokens and latents and prioritize important input regions. By training with random dropping of tail latents, ELIT learns to produce importance-ordered representations with earlier latents capturing global structure while later ones contain information to refine details. At inference, the number of latents can be dynamically adjusted to match compute constraints. ELIT is deliberately minimal, adding two cross-attention layers while leaving the rectified flow objective and the DiT stack unchanged. Across datasets and architectures (DiT, U-ViT, HDiT, MM-DiT), ELIT delivers consistent gains. On ImageNet-1K 512px, ELIT delivers an average gain of 35.3% and 39.6% in FID and FDD scores. Project page: https://snap-research.github.io/elit/
Elastic Latent Interface Transformer (ELIT) decouples compute from image resolution in diffusion transformers by introducing learnable latent tokens that adaptively prioritize important regions, enabling dynamic resource allocation without modifying core model architecture.
diffusion transformers FLOPs latent interface transformer blocks cross-attention layers spatial tokens rectified flow objective DiT U-ViT HDiT MM-DiT FID FDD