MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models
Post-training quantization (PTQ) with computational invariance for Large Language Models~(LLMs) have demonstrated remarkable advances, however, their application to Multimodal Large Language Models~(MLLMs) presents substantial challenges. In this paper, we analyze SmoothQuant as a case study and identify two critical issues: Smoothing Misalignment and Cross-Modal Computational Invariance. To address these issues, we propose Modality-Aware Smoothing Quantization (MASQuant), a novel framework that introduces (1) Modality-Aware Smoothing (MAS), which learns separate, modality-specific smoothing factors to prevent Smoothing Misalignment, and (2) Cross-Modal Compensation (CMC), which addresses Cross-modal Computational Invariance by using SVD whitening to transform multi-modal activation differences into low-rank forms, enabling unified quantization across modalities. MASQuant demonstrates stable quantization performance across both dual-modal and tri-modal MLLMs. Experimental results show that MASQuant is competitive among the state-of-the-art PTQ algorithms. Source code: https://github.com/alibaba/EfficientAI.
69aa9c06329356be7a655bc4
Lulu Hu
69aa9c06329356be7a655bc5
Wenhu Xiao
claimed_verified
2026-03-06T13:54:32.077Z
64c781f67b4d0d947cb4e29a
/avatars/ac40741a42442a9fbb94a47c8ab254b1.svg
wenhu
tigerxiao
user
tigerxiao
69aa9c06329356be7a655bc6
Xin Chen
69aa9c06329356be7a655bc7
Xinhua Xu
69aa9c06329356be7a655bc8
Bowen Xu
claimed_verified
2026-03-06T13:54:30.118Z
6418142857c3a491a4d64f1f
/avatars/85db855cdc65ef9e3fd20d15e50ef1cc.svg
bowen xu
bowiehsu
user
bowiehsu
69aa9c06329356be7a655bc9
Kun Li
claimed_verified
2026-03-12T09:08:00.646Z
6615ffad24c94fd8a224b1e4
/avatars/8b5b9242a251f74ac6e6db3b5fb235ec.svg
Kun Li
pearcry
user
pearcry
69aa9c06329356be7a655bca
Yongliang Tao
MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models
Post-training quantization (PTQ) with computational invariance for Large Language Models~(LLMs) have demonstrated remarkable advances, however, their application to Multimodal Large Language Models~(MLLMs) presents substantial challenges. In this paper, we analyze SmoothQuant as a case study and identify two critical issues: Smoothing Misalignment and Cross-Modal Computational Invariance. To address these issues, we propose Modality-Aware Smoothing Quantization (MASQuant), a novel framework that introduces (1) Modality-Aware Smoothing (MAS), which learns separate, modality-specific smoothing factors to prevent Smoothing Misalignment, and (2) Cross-Modal Compensation (CMC), which addresses Cross-modal Computational Invariance by using SVD whitening to transform multi-modal activation differences into low-rank forms, enabling unified quantization across modalities. MASQuant demonstrates stable quantization performance across both dual-modal and tri-modal MLLMs. Experimental results show that MASQuant is competitive among the state-of-the-art PTQ algorithms. Source code: https://github.com/alibaba/EfficientAI.
Post-training quantization for multimodal large language models addresses smoothing misalignment and cross-modal computational invariance through modality-aware smoothing and cross-modal compensation techniques.
Post-training quantization Large Language Models Multimodal Large Language Models SmoothQuant Smoothing Misalignment Cross-Modal Computational Invariance Modality-Aware Smoothing Cross-Modal Compensation SVD whitening quantization