Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation
A recent cutting-edge topic in multimodal modeling is to unify visual comprehension and generation within a single model. However, the two tasks demand mismatched decoding regimes and visual representations, making it non-trivial to jointly optimize within a shared feature space. In this work, we present Cheers, a unified multimodal model that decouples patch-level details from semantic representations, thereby stabilizing semantics for multimodal understanding and improving fidelity for image generation via gated detail residuals. Cheers includes three key components: (i) a unified vision tokenizer that encodes and compresses image latent states into semantic tokens for efficient LLM conditioning, (ii) an LLM-based Transformer that unifies autoregressive decoding for text generation and diffusion decoding for image generation, and (iii) a cascaded flow matching head that decodes visual semantics first and then injects semantically gated detail residuals from the vision tokenizer to refine high-frequency content. Experiments on popular benchmarks demonstrate that Cheers matches or surpasses advanced UMMs in both visual understanding and generation. Cheers also achieves 4x token compression, enabling more efficient high-resolution image encoding and generation. Notably, Cheers outperforms the Tar-1.5B on the popular benchmarks GenEval and MMBench, while requiring only 20% of the training cost, indicating effective and efficient (i.e., 4x token compression) unified multimodal modeling. We will release all code and data for future research.
69b75dcb91a28162b233fbbd
Yichen Zhang
69b75dcb91a28162b233fbbe
Da Peng
claimed_verified
2026-03-16T10:40:21.732Z
665b276ea132595fafc4a431
/avatars/f238582a506bc24a822360b4d4cbd625.svg
PengDa
PengDa02
user
PengDa02
69b75dcb91a28162b233fbbf
Zonghao Guo
admin_assigned
2026-03-16T10:53:56.321Z
6491af36c1741666238f3bff
/avatars/0ee7d2ec1566e2cc5e8f144140e17f00.svg
Zonghao Guo
guozonghao96
user
guozonghao96
69b75dcb91a28162b233fbc0
Zijian Zhang
69b75dcb91a28162b233fbc1
Xuesong Yang
69b75dcb91a28162b233fbc2
Tong Sun
69b75dcb91a28162b233fbc3
Shichu Sun
69b75dcb91a28162b233fbc4
Yidan Zhang
69b75dcb91a28162b233fbc5
Yanghao Li
admin_assigned
2026-03-16T10:54:24.763Z
65dad3870af7e21ba473439f
/avatars/da542e7d68ae937bbdb791f17096bb1c.svg
Yanghao Li
FrozzZen
user
FrozzZen
69b75dcb91a28162b233fbc6
Haiyan Zhao
69b75dcb91a28162b233fbc7
Wang Xu
69b75dcb91a28162b233fbc8
Qi Shi
69b75dcb91a28162b233fbc9
Yangang Sun
admin_assigned
2026-03-16T10:54:35.759Z
66b07a3c5a4b8b3438ee6d1d
/avatars/224cf97a9ede71e4a600d6fb91971039.svg
Yangang Sun
CLSN-S
user
CLSN-S
69b75dcb91a28162b233fbca
Chi Chen
69b75dcb91a28162b233fbcb
Shuo Wang
69b75dcb91a28162b233fbcc
Yukun Yan
69b75dcb91a28162b233fbcd
Xu Han
69b75dcb91a28162b233fbce
Qiang Ma
69b75dcb91a28162b233fbcf
Wei Ke
69b75dcb91a28162b233fbd0
Liang Wang
69b75dcb91a28162b233fbd1
Zhiyuan Liu
69b75dcb91a28162b233fbd2
Maosong Sun
Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation
A recent cutting-edge topic in multimodal modeling is to unify visual comprehension and generation within a single model. However, the two tasks demand mismatched decoding regimes and visual representations, making it non-trivial to jointly optimize within a shared feature space. In this work, we present Cheers, a unified multimodal model that decouples patch-level details from semantic representations, thereby stabilizing semantics for multimodal understanding and improving fidelity for image generation via gated detail residuals. Cheers includes three key components: (i) a unified vision tokenizer that encodes and compresses image latent states into semantic tokens for efficient LLM conditioning, (ii) an LLM-based Transformer that unifies autoregressive decoding for text generation and diffusion decoding for image generation, and (iii) a cascaded flow matching head that decodes visual semantics first and then injects semantically gated detail residuals from the vision tokenizer to refine high-frequency content. Experiments on popular benchmarks demonstrate that Cheers matches or surpasses advanced UMMs in both visual understanding and generation. Cheers also achieves 4x token compression, enabling more efficient high-resolution image encoding and generation. Notably, Cheers outperforms the Tar-1.5B on the popular benchmarks GenEval and MMBench, while requiring only 20% of the training cost, indicating effective and efficient (i.e., 4x token compression) unified multimodal modeling. We will release all code and data for future research.
Cheers is a unified multimodal model that decouples visual details from semantic representations using a vision tokenizer, LLM-based Transformer, and cascaded flow matching head to achieve efficient joint optimization for both visual understanding and generation tasks.
unified multimodal model vision tokenizer semantic tokens LLM-based Transformer autoregressive decoding diffusion decoding cascaded flow matching head visual comprehension visual generation token compression