HybridStitch: Pixel and Timestep Level Model Stitching for Diffusion Acceleration
Diffusion models have demonstrated a remarkable ability in Text-to-Image (T2I) generation applications. Despite the advanced generation output, they suffer from heavy computation overhead, especially for large models that contain tens of billions of parameters. Prior work has illustrated that replacing part of the denoising steps with a smaller model still maintains the generation quality. However, these methods only focus on saving computation for some timesteps, ignoring the difference in compute demand within one timestep. In this work, we propose HybridStitch, a new T2I generation paradigm that treats generation like editing. Specifically, we introduce a hybrid stage that jointly incorporates both the large model and the small model. HybridStitch separates the entire image into two regions: one that is relatively easy to render, enabling an early transition to the smaller model, and another that is more complex and therefore requires refinement by the large model. HybridStitch employs the small model to construct a coarse sketch while exploiting the large model to edit and refine the complex regions. According to our evaluation, HybridStitch achieves 1.83times speedup on Stable Diffusion 3, which is faster than all existing mixture of model methods.
69b773e891a28162b233fc26
Desen Sun
admin_assigned
2026-03-16T11:10:08.873Z
67046563fa3ab087c5b8193d
/avatars/569dc9043c0dd130d9ed39939fca6f59.svg
Desen Sun
desensun
user
desensun
69b773e891a28162b233fc27
Jason Hon
claimed_verified
2026-03-16T10:39:59.398Z
6917920b7bf33612eb9781d4
/avatars/2120a2a6124eed6e547edf94e58c83d2.svg
Jason Hon
JasonH53
user
JasonH53
69b773e891a28162b233fc28
Jintao Zhang
claimed_verified
2026-03-16T10:40:01.612Z
66c0a08bac74db25de8427ec
https://cdn-avatars.huggingface.co/v1/production/uploads/66c0a08bac74db25de8427ec/9D-piDBZqSt6KNkHImmkv.jpeg
Jintao Zhang
jt-zhang
user
jt-zhang
69b773e891a28162b233fc29
Sihang Liu
HybridStitch: Pixel and Timestep Level Model Stitching for Diffusion Acceleration
Diffusion models have demonstrated a remarkable ability in Text-to-Image (T2I) generation applications. Despite the advanced generation output, they suffer from heavy computation overhead, especially for large models that contain tens of billions of parameters. Prior work has illustrated that replacing part of the denoising steps with a smaller model still maintains the generation quality. However, these methods only focus on saving computation for some timesteps, ignoring the difference in compute demand within one timestep. In this work, we propose HybridStitch, a new T2I generation paradigm that treats generation like editing. Specifically, we introduce a hybrid stage that jointly incorporates both the large model and the small model. HybridStitch separates the entire image into two regions: one that is relatively easy to render, enabling an early transition to the smaller model, and another that is more complex and therefore requires refinement by the large model. HybridStitch employs the small model to construct a coarse sketch while exploiting the large model to edit and refine the complex regions. According to our evaluation, HybridStitch achieves 1.83times speedup on Stable Diffusion 3, which is faster than all existing mixture of model methods.
HybridStitch enables faster text-to-image generation by combining large and small models to efficiently handle different image complexity regions.
diffusion models text-to-image generation denoising steps parameter-efficient fine-tuning hybrid stage coarse sketch fine-grained refinement