Glance: Accelerating Diffusion Models with 1 Sample
Diffusion models have achieved remarkable success in image generation, yet their deployment remains constrained by the heavy computational cost and the need for numerous inference steps. Previous efforts on fewer-step distillation attempt to skip redundant steps by training compact student models, yet they often suffer from heavy retraining costs and degraded generalization. In this work, we take a different perspective: we accelerate smartly, not evenly, applying smaller speedups to early semantic stages and larger ones to later redundant phases. We instantiate this phase-aware strategy with two experts that specialize in slow and fast denoising phases. Surprisingly, instead of investing massive effort in retraining student models, we find that simply equipping the base model with lightweight LoRA adapters achieves both efficient acceleration and strong generalization. We refer to these two adapters as Slow-LoRA and Fast-LoRA. Through extensive experiments, our method achieves up to 5 acceleration over the base model while maintaining comparable visual quality across diverse benchmarks. Remarkably, the LoRA experts are trained with only 1 samples on a single V100 within one hour, yet the resulting models generalize strongly on unseen prompts.
692fad4226742347f61dac42
Zhuobai Dong
692fad4226742347f61dac43
Rui Zhao
claimed_verified
2025-12-15T08:14:18.113Z
652b83b73b5997ed71a310f2
https://cdn-avatars.huggingface.co/v1/production/uploads/652b83b73b5997ed71a310f2/ipCpdeHUp4-0OmRz5z8IW.png
Rui Zhao
ruizhaocv
user
ruizhaocv
692fad4226742347f61dac44
Songjie Wu
692fad4226742347f61dac45
Junchao Yi
claimed_verified
2026-04-09T08:08:20.568Z
67ee16793431887c4ddeeb25
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/qYNhJKGo5oZCegA97dk9R.png
Junchao Yi
Junc1i
user
Junc1i
692fad4226742347f61dac46
Linjie Li
692fad4226742347f61dac47
Zhengyuan Yang
692fad4226742347f61dac48
Lijuan Wang
692fad4226742347f61dac49
Alex Jinpeng Wang
Glance: Accelerating Diffusion Models with 1 Sample
Diffusion models have achieved remarkable success in image generation, yet their deployment remains constrained by the heavy computational cost and the need for numerous inference steps. Previous efforts on fewer-step distillation attempt to skip redundant steps by training compact student models, yet they often suffer from heavy retraining costs and degraded generalization. In this work, we take a different perspective: we accelerate smartly, not evenly, applying smaller speedups to early semantic stages and larger ones to later redundant phases. We instantiate this phase-aware strategy with two experts that specialize in slow and fast denoising phases. Surprisingly, instead of investing massive effort in retraining student models, we find that simply equipping the base model with lightweight LoRA adapters achieves both efficient acceleration and strong generalization. We refer to these two adapters as Slow-LoRA and Fast-LoRA. Through extensive experiments, our method achieves up to 5 acceleration over the base model while maintaining comparable visual quality across diverse benchmarks. Remarkably, the LoRA experts are trained with only 1 samples on a single V100 within one hour, yet the resulting models generalize strongly on unseen prompts.
Using phase-aware LoRA adapters, diffusion models achieve efficient acceleration and strong generalization with minimal retraining.
diffusion models computational cost inference steps distillation compact student models retraining costs phase-aware strategy slow and fast denoising phases lightweight LoRA adapters Slow-LoRA Fast-LoRA acceleration visual quality benchmarks samples V100 unseen prompts