SoundWeaver: Semantic Warm-Starting for Text-to-Audio Diffusion Serving
Text-to-audio diffusion models produce high-fidelity audio but require tens of function evaluations (NFEs), incurring multi-second latency and limited throughput. We present SoundWeaver, the first training-free, model-agnostic serving system that accelerates text-to-audio diffusion by warm-starting from semantically similar cached audio. SoundWeaver introduces three components: a Reference Selector that retrieves and temporally aligns cached candidates via semantic and duration-aware gating; a Skip Gater that dynamically determines the percentage of NFEs to skip; and a lightweight Cache Manager that maintains cache utility through quality-aware eviction and refinement. On real-world audio traces, SoundWeaver achieves 1.8--3.0 times  latency reduction with a cache of only {sim}1K entries while preserving or improving perceptual quality.
69b0a2971f7902d47f6df8e1
Ayush Barik
claimed_verified
2026-03-11T08:44:48.555Z
672e6d23bcc344d5048a9c65
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ZINVVKMXVgbSc-eiYD9WQ.png
Ayush Barik
jormungandr2017
user
jormungandr2017
69b0a2971f7902d47f6df8e2
Sofia Stoica
69b0a2971f7902d47f6df8e3
Nikhil Sarda
69b0a2971f7902d47f6df8e4
Arnav Kethana
69b0a2971f7902d47f6df8e5
Abhinav Khanduja
69b0a2971f7902d47f6df8e6
Muchen Xu
69b0a2971f7902d47f6df8e7
Fan Lai
SoundWeaver: Semantic Warm-Starting for Text-to-Audio Diffusion Serving
Text-to-audio diffusion models produce high-fidelity audio but require tens of function evaluations (NFEs), incurring multi-second latency and limited throughput. We present SoundWeaver, the first training-free, model-agnostic serving system that accelerates text-to-audio diffusion by warm-starting from semantically similar cached audio. SoundWeaver introduces three components: a Reference Selector that retrieves and temporally aligns cached candidates via semantic and duration-aware gating; a Skip Gater that dynamically determines the percentage of NFEs to skip; and a lightweight Cache Manager that maintains cache utility through quality-aware eviction and refinement. On real-world audio traces, SoundWeaver achieves 1.8--3.0 times  latency reduction with a cache of only {sim}1K entries while preserving or improving perceptual quality.
SoundWeaver accelerates text-to-audio diffusion generation by caching semantically similar audio and dynamically skipping function evaluations, achieving significant latency reduction with minimal quality loss.
text-to-audio diffusion models function evaluations latency throughput warm-starting semantic similarity cached audio Reference Selector Skip Gater Cache Manager temporal alignment quality-aware eviction refinement