BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection
The exponential expansion of context windows in LLMs has unlocked capabilities for long-document understanding but introduced severe bottlenecks in inference latency and information utilization. Existing compression methods often suffer from high training costs or semantic fragmentation due to aggressive token pruning. In this paper, we propose BEAVER, a novel training-free framework that shifts compression from linear token removal to structure-aware hierarchical selection. BEAVER maximizes hardware parallelism by mapping variable-length contexts into dense page-level tensors via dual-path pooling, and preserves discourse integrity through a hybrid planner combining semantic and lexical dual-branch selection with sentence smoothing. Extensive evaluations on four long-context benchmarks demonstrate that BEAVER achieves comparable performance to state-of-the-art (SOTA) methods like LongLLMLingua. Notably, on the RULER benchmark, BEAVER maintains high fidelity in multi-needle retrieval where baselines deteriorate. Regarding efficiency, BEAVER reduces latency by 26.4x on 128k contexts, offering a scalable solution for high-throughput applications. Our code is available at https://cslikai.cn/BEAVER/.
69c0b13b3fd3fef6237a6869
Zhengpei Hu
69c0b13b3fd3fef6237a686a
Kai Li
claimed_verified
2026-03-23T09:16:06.712Z
6387676c23da90491eb9fb16
https://cdn-avatars.huggingface.co/v1/production/uploads/1669818175965-noauth.jpeg
Kai Li
JusperLee
user
JusperLee
69c0b13b3fd3fef6237a686b
Dapeng Fu
69c0b13b3fd3fef6237a686c
Chang Zeng
claimed_verified
2026-03-23T09:16:04.791Z
62597cb0c53775eff9eba55f
/avatars/1e8cd225cac80fb91583d1a59da58aea.svg
Chang Zeng
woshinibaba
user
woshinibaba
69c0b13b3fd3fef6237a686d
Yue Li
69c0b13b3fd3fef6237a686e
Yuanhao Tang
69c0b13b3fd3fef6237a686f
Jianqiang Huang
BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection
The exponential expansion of context windows in LLMs has unlocked capabilities for long-document understanding but introduced severe bottlenecks in inference latency and information utilization. Existing compression methods often suffer from high training costs or semantic fragmentation due to aggressive token pruning. In this paper, we propose BEAVER, a novel training-free framework that shifts compression from linear token removal to structure-aware hierarchical selection. BEAVER maximizes hardware parallelism by mapping variable-length contexts into dense page-level tensors via dual-path pooling, and preserves discourse integrity through a hybrid planner combining semantic and lexical dual-branch selection with sentence smoothing. Extensive evaluations on four long-context benchmarks demonstrate that BEAVER achieves comparable performance to state-of-the-art (SOTA) methods like LongLLMLingua. Notably, on the RULER benchmark, BEAVER maintains high fidelity in multi-needle retrieval where baselines deteriorate. Regarding efficiency, BEAVER reduces latency by 26.4x on 128k contexts, offering a scalable solution for high-throughput applications. Our code is available at https://cslikai.cn/BEAVER/.
BEAVER is a training-free framework that improves long-context LLM inference by using structure-aware hierarchical selection and dense tensor mapping to reduce latency while maintaining semantic integrity.
context windows long-document understanding inference latency token pruning training-free framework hierarchical selection dual-path pooling discourse integrity semantic dual-branch selection lexical dual-branch selection sentence smoothing long-context benchmarks multi-needle retrieval latency reduction high-throughput applications