BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
Large language model (LLM) inference is often bounded by memory footprint and memory bandwidth in resource-constrained deployments, making quantization a fundamental technique for efficient serving. While post-training quantization (PTQ) maintains high fidelity at 4-bit, it deteriorates at 2-3 bits. Fundamentally, existing methods enforce a shape-invariant quantization grid (e.g., the fixed uniform intervals of UINT2) for each group, severely restricting the feasible set for error minimization. To address this, we propose Bit-Plane Decomposition Quantization (BPDQ), which constructs a variable quantization grid via bit-planes and scalar coefficients, and iteratively refines them using approximate second-order information while progressively compensating quantization errors to minimize output discrepancy. In the 2-bit regime, BPDQ enables serving Qwen2.5-72B on a single RTX 3090 with 83.85% GSM8K accuracy (vs. 90.83% at 16-bit). Moreover, we provide theoretical analysis showing that the variable grid expands the feasible set, and that the quantization process consistently aligns with the optimization objective in Hessian-induced geometry. Code: github.com/KingdalfGoodman/BPDQ.
6992963950fb2c0be4783970
Junyu Chen
6992963950fb2c0be4783971
Jungang Li
claimed_verified
2026-02-17T15:53:28.292Z
64b76528fdb702b3d8641514
https://cdn-avatars.huggingface.co/v1/production/uploads/64b76528fdb702b3d8641514/Ho-uWcQCAEIURM1lhWEWJ.jpeg
Jungang Li
Jungang
user
Jungang
6992963950fb2c0be4783972
Jing Xiong
6992963950fb2c0be4783973
Wenjie Wang
6992963950fb2c0be4783974
Qingyao Yang
6992963950fb2c0be4783975
He Xiao
claimed_verified
2026-03-19T13:04:53.084Z
671c6ade11009e5ac0d5e217
/avatars/c7c837807afd3bf7b2350c2f9f205201.svg
He Xiao
River555
user
River555
6992963950fb2c0be4783976
Zhen Li
6992963950fb2c0be4783977
Taiqiang Wu
claimed_verified
2026-02-25T17:32:35.334Z
6621cea88850e38ffbb1854f
https://cdn-avatars.huggingface.co/v1/production/uploads/6621cea88850e38ffbb1854f/qmKHts7YCUylxqzPHk4uT.jpeg
Taki WU
taki555
user
taki555
6992963950fb2c0be4783978
Mengzhao Chen
6992963950fb2c0be4783979
Zhen Peng
6992963950fb2c0be478397a
Chaofan Tao
6992963950fb2c0be478397b
Long Shi
6992963950fb2c0be478397c
Hongxia Yang
6992963950fb2c0be478397d
Ngai Wong
BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
Large language model (LLM) inference is often bounded by memory footprint and memory bandwidth in resource-constrained deployments, making quantization a fundamental technique for efficient serving. While post-training quantization (PTQ) maintains high fidelity at 4-bit, it deteriorates at 2-3 bits. Fundamentally, existing methods enforce a shape-invariant quantization grid (e.g., the fixed uniform intervals of UINT2) for each group, severely restricting the feasible set for error minimization. To address this, we propose Bit-Plane Decomposition Quantization (BPDQ), which constructs a variable quantization grid via bit-planes and scalar coefficients, and iteratively refines them using approximate second-order information while progressively compensating quantization errors to minimize output discrepancy. In the 2-bit regime, BPDQ enables serving Qwen2.5-72B on a single RTX 3090 with 83.85% GSM8K accuracy (vs. 90.83% at 16-bit). Moreover, we provide theoretical analysis showing that the variable grid expands the feasible set, and that the quantization process consistently aligns with the optimization objective in Hessian-induced geometry. Code: github.com/KingdalfGoodman/BPDQ.
Bit-Plane Decomposition Quantization (BPDQ) improves low-bit quantization by using variable quantization grids derived from bit-planes and scalar coefficients, achieving better accuracy than traditional methods in resource-constrained LLM inference.
post-training quantization quantization grid bit-plane decomposition scalar coefficients second-order information quantization error Hessian-induced geometry