SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing
Diffusion Transformers (DiTs) have become a leading backbone for video generation, yet their quadratic attention cost remains a major bottleneck. Sparse attention reduces this cost by computing only a subset of attention blocks. However, prior methods often either drop the remaining blocks, which incurs information loss, or rely on learned predictors to approximate them, introducing training overhead and potential output distribution shifting. In this paper, we show that the missing contributions can be recovered without training: after semantic clustering, keys and values within each block exhibit strong similarity and can be well summarized by a small set of cluster centroids. Based on this observation, we introduce SVG-EAR, a parameter-free linear compensation branch that uses the centroid to approximate skipped blocks and recover their contributions. While centroid compensation is accurate for most blocks, it can fail on a small subset. Standard sparsification typically selects blocks by attention scores, which indicate where the model places its attention mass, but not where the approximation error would be largest. SVG-EAR therefore performs error-aware routing: a lightweight probe estimates the compensation error for each block, and we compute exactly the blocks with the highest error-to-cost ratio while compensating for skipped blocks. We provide theoretical guarantees that relate attention reconstruction error to clustering quality, and empirically show that SVG-EAR improves the quality-efficiency trade-off and increases throughput at the same generation fidelity on video diffusion tasks. Overall, SVG-EAR establishes a clear Pareto frontier over prior approaches, achieving up to 1.77times and 1.93times speedups while maintaining PSNRs of up to 29.759 and 31.043 on Wan2.2 and HunyuanVideo, respectively.
69b0c49c1f7902d47f6df93c
Xuanyi Zhou
69b0c49c1f7902d47f6df93d
Qiuyang Mang
69b0c49c1f7902d47f6df93e
Shuo Yang
claimed_verified
2026-03-11T08:44:22.242Z
642b970ceb31218a5f204a29
/avatars/582287f477bbb1a0842787145e375fd3.svg
andy-yang
andy-yang
user
andy-yang
69b0c49c1f7902d47f6df93f
Haocheng Xi
69b0c49c1f7902d47f6df940
Jintao Zhang
claimed_verified
2026-03-11T08:44:20.465Z
66c0a08bac74db25de8427ec
https://cdn-avatars.huggingface.co/v1/production/uploads/66c0a08bac74db25de8427ec/9D-piDBZqSt6KNkHImmkv.jpeg
Jintao Zhang
jt-zhang
user
jt-zhang
69b0c49c1f7902d47f6df941
Huanzhi Mao
69b0c49c1f7902d47f6df942
Joseph E. Gonzalez
69b0c49c1f7902d47f6df943
Kurt Keutzer
69b0c49c1f7902d47f6df944
Ion Stoica
69b0c49c1f7902d47f6df945
Alvin Cheung
SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing
Diffusion Transformers (DiTs) have become a leading backbone for video generation, yet their quadratic attention cost remains a major bottleneck. Sparse attention reduces this cost by computing only a subset of attention blocks. However, prior methods often either drop the remaining blocks, which incurs information loss, or rely on learned predictors to approximate them, introducing training overhead and potential output distribution shifting. In this paper, we show that the missing contributions can be recovered without training: after semantic clustering, keys and values within each block exhibit strong similarity and can be well summarized by a small set of cluster centroids. Based on this observation, we introduce SVG-EAR, a parameter-free linear compensation branch that uses the centroid to approximate skipped blocks and recover their contributions. While centroid compensation is accurate for most blocks, it can fail on a small subset. Standard sparsification typically selects blocks by attention scores, which indicate where the model places its attention mass, but not where the approximation error would be largest. SVG-EAR therefore performs error-aware routing: a lightweight probe estimates the compensation error for each block, and we compute exactly the blocks with the highest error-to-cost ratio while compensating for skipped blocks. We provide theoretical guarantees that relate attention reconstruction error to clustering quality, and empirically show that SVG-EAR improves the quality-efficiency trade-off and increases throughput at the same generation fidelity on video diffusion tasks. Overall, SVG-EAR establishes a clear Pareto frontier over prior approaches, achieving up to 1.77times and 1.93times speedups while maintaining PSNRs of up to 29.759 and 31.043 on Wan2.2 and HunyuanVideo, respectively.
SVG-EAR introduces a parameter-free method for video diffusion transformers that recovers missing attention contributions through centroid-based approximation and error-aware routing for improved efficiency.
diffusion transformers sparse attention attention cost semantic clustering cluster centroids centroid compensation error-aware routing attention reconstruction error Pareto frontier