Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compression
The deployment of Large Language Models is constrained by the memory and bandwidth demands of static weights and dynamic Key-Value cache. SVD-based compression provides a hardware-friendly solution to reduce these costs. However, existing methods suffer from two key limitations: some are suboptimal in reconstruction error, while others are theoretically optimal but practically inefficient. In this paper, we propose Swift-SVD, an activation-aware, closed-form compression framework that simultaneously guarantees theoretical optimum, practical efficiency and numerical stability. Swift-SVD incrementally aggregates covariance of output activations given a batch of inputs and performs a single eigenvalue decomposition after aggregation, enabling training-free, fast, and optimal layer-wise low-rank approximation. We employ effective rank to analyze local layer-wise compressibility and design a dynamic rank allocation strategy that jointly accounts for local reconstruction loss and end-to-end layer importance. Extensive experiments across six LLMs and eight datasets demonstrate that Swift-SVD outperforms state-of-the-art baselines, achieving optimal compression accuracy while delivering 3-70X speedups in end-to-end compression time. Our code will be released upon acceptance.
69d3d0542c523d36431fae90
Ruoling Qi
claimed_verified
2026-04-07T08:37:55.783Z
65fc48f7187c4ac021092434
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/PkEwEqltrykC3YZE6pEaD.jpeg
齐若伶
heihei760
user
heihei760
69d3d0542c523d36431fae91
Yirui Liu
69d3d0542c523d36431fae92
Xuaner Wu
69d3d0542c523d36431fae93
Xiangyu Wang
69d3d0542c523d36431fae94
Ming Li
69d3d0542c523d36431fae95
Chen Chen
69d3d0542c523d36431fae96
Jian Chen
claimed_verified
2026-04-07T08:37:59.392Z
64593836c16ecb4815e082ce
https://cdn-avatars.huggingface.co/v1/production/uploads/64593836c16ecb4815e082ce/cHv6bOowt4N7n-dweNozO.jpeg
Jian Chen
puar-playground
user
puar-playground
69d3d0542c523d36431fae97
Yin Chen
69d3d0542c523d36431fae98
Qizhen Weng
Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compression
The deployment of Large Language Models is constrained by the memory and bandwidth demands of static weights and dynamic Key-Value cache. SVD-based compression provides a hardware-friendly solution to reduce these costs. However, existing methods suffer from two key limitations: some are suboptimal in reconstruction error, while others are theoretically optimal but practically inefficient. In this paper, we propose Swift-SVD, an activation-aware, closed-form compression framework that simultaneously guarantees theoretical optimum, practical efficiency and numerical stability. Swift-SVD incrementally aggregates covariance of output activations given a batch of inputs and performs a single eigenvalue decomposition after aggregation, enabling training-free, fast, and optimal layer-wise low-rank approximation. We employ effective rank to analyze local layer-wise compressibility and design a dynamic rank allocation strategy that jointly accounts for local reconstruction loss and end-to-end layer importance. Extensive experiments across six LLMs and eight datasets demonstrate that Swift-SVD outperforms state-of-the-art baselines, achieving optimal compression accuracy while delivering 3-70X speedups in end-to-end compression time. Our code will be released upon acceptance.
Swift-SVD is a compression framework that achieves optimal low-rank approximations for large language models through efficient covariance aggregation and eigenvalue decomposition, enabling faster and more accurate model compression.
SVD-based compression activation-aware compression closed-form compression eigenvalue decomposition low-rank approximation dynamic rank allocation effective rank local layer-wise compressibility