KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs
Large language models rely on kv-caches to avoid redundant computation during autoregressive decoding, but as context length grows, reading and writing the cache can quickly saturate GPU memory bandwidth. Recent work has explored KV-cache compression, yet most approaches neglect the data-dependent nature of kv-caches and their variation across layers. We introduce KV-CoRE KV-cache Compressibility by Rank Evaluation), an SVD-based method for quantifying the data-dependent low-rank compressibility of kv-caches. KV-CoRE computes the optimal low-rank approximation under the Frobenius norm and, being gradient-free and incremental, enables efficient dataset-level, layer-wise evaluation. Using this method, we analyze multiple models and datasets spanning five English domains and sixteen languages, uncovering systematic patterns that link compressibility to model architecture, training data, and language coverage. As part of this analysis, we employ the Normalized Effective Rank as a metric of compressibility and show that it correlates strongly with performance degradation under compression. Our study establishes a principled evaluation framework and the first large-scale benchmark of kv-cache compressibility in LLMs, offering insights for dynamic, data-aware compression and data-centric model development.
698b3b891b2dc6b37d61b4cc
Jian Chen
claimed_verified
2026-02-11T11:16:51.465Z
64593836c16ecb4815e082ce
https://cdn-avatars.huggingface.co/v1/production/uploads/64593836c16ecb4815e082ce/cHv6bOowt4N7n-dweNozO.jpeg
Jian Chen
puar-playground
user
puar-playground
698b3b891b2dc6b37d61b4cd
Zhuoran Wang
698b3b891b2dc6b37d61b4ce
Jiayu Qin
698b3b891b2dc6b37d61b4cf
Ming Li
698b3b891b2dc6b37d61b4d0
Meng Wang
698b3b891b2dc6b37d61b4d1
Changyou Chen
698b3b891b2dc6b37d61b4d2
Yin Chen
698b3b891b2dc6b37d61b4d3
Qizhen Weng
698b3b891b2dc6b37d61b4d4
Yirui Liu
KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs
Large language models rely on kv-caches to avoid redundant computation during autoregressive decoding, but as context length grows, reading and writing the cache can quickly saturate GPU memory bandwidth. Recent work has explored KV-cache compression, yet most approaches neglect the data-dependent nature of kv-caches and their variation across layers. We introduce KV-CoRE KV-cache Compressibility by Rank Evaluation), an SVD-based method for quantifying the data-dependent low-rank compressibility of kv-caches. KV-CoRE computes the optimal low-rank approximation under the Frobenius norm and, being gradient-free and incremental, enables efficient dataset-level, layer-wise evaluation. Using this method, we analyze multiple models and datasets spanning five English domains and sixteen languages, uncovering systematic patterns that link compressibility to model architecture, training data, and language coverage. As part of this analysis, we employ the Normalized Effective Rank as a metric of compressibility and show that it correlates strongly with performance degradation under compression. Our study establishes a principled evaluation framework and the first large-scale benchmark of kv-cache compressibility in LLMs, offering insights for dynamic, data-aware compression and data-centric model development.
KV-CoRE method evaluates kv-cache compressibility through SVD-based low-rank approximation, revealing patterns linking compressibility to model architecture and training data across multiple languages and domains.
KV-cache SVD-based method low-rank approximation Frobenius norm gradient-free incremental evaluation Normalized Effective Rank compressibility autoregressive decoding GPU memory bandwidth KV-cache compression dataset-level evaluation layer-wise evaluation