SignRoundV2: Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs
Extreme low-bit quantization is critical for efficiently deploying Large Language Models (LLMs), yet it often leads to severe performance degradation at 2-bits and even 4-bits (e.g., MXFP4). We present SignRoundV2, a post-training quantization framework that is highly effective even without mixed-precision. SignRoundV2 introduces (1) a fast sensitivity metric that combines gradient information with quantization-induced deviations to guide layer-wise bit allocation, and (2) a lightweight pre-tuning search for quantization scales to improve extremely low-bit quantization. These components allow SignRoundV2 to close the gap with full-precision models. Extensive experiments indicate that our method sustains competitive accuracy for LLMs, achieving production-grade performance with about 1 percent variance at 4-5 bits and strong results even at 2 bits. The implementation is available at https://github.com/intel/auto-round.
693245586d1060ca587a2612
Wenhua Cheng
claimed_verified
2025-12-05T08:29:48.262Z
634772780a9fa1ac49994510
/avatars/2e88e32ac0d45a1c624026e497eb00b3.svg
wenhua cheng
wenhuach
user
wenhuach
693245586d1060ca587a2613
Weiwei Zhang
693245586d1060ca587a2614
Heng Guo
claimed_verified
2025-12-05T15:14:59.807Z
6539b8e6e279d2fc80bf438b
/avatars/3a47a0ce8caaa38ddc2e0e43ac722203.svg
Heng Guo
n1ck-guo
user
n1ck-guo
693245586d1060ca587a2615
Haihao Shen
claimed_verified
2025-12-05T08:29:50.577Z
60ac3318e3de7c7440abb850
https://cdn-avatars.huggingface.co/v1/production/uploads/60ac3318e3de7c7440abb850/DkMrPBr6Ew_dS_c9kog4e.jpeg
Haihao Shen
Haihao
user
Haihao
SignRoundV2: Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs
Extreme low-bit quantization is critical for efficiently deploying Large Language Models (LLMs), yet it often leads to severe performance degradation at 2-bits and even 4-bits (e.g., MXFP4). We present SignRoundV2, a post-training quantization framework that is highly effective even without mixed-precision. SignRoundV2 introduces (1) a fast sensitivity metric that combines gradient information with quantization-induced deviations to guide layer-wise bit allocation, and (2) a lightweight pre-tuning search for quantization scales to improve extremely low-bit quantization. These components allow SignRoundV2 to close the gap with full-precision models. Extensive experiments indicate that our method sustains competitive accuracy for LLMs, achieving production-grade performance with about 1 percent variance at 4-5 bits and strong results even at 2 bits. The implementation is available at https://github.com/intel/auto-round.
SignRoundV2, a post-training quantization framework, achieves competitive accuracy for Large Language Models at extremely low-bit quantization through layer-wise bit allocation and pre-tuning scale search.
post-training quantization SignRoundV2 gradient information quantization-induced deviations layer-wise bit allocation pre-tuning Large Language Models low-bit quantization mixed-precision production-grade performance