UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations
Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap. However, discretization inevitably discards fine-grained semantic information, leading to suboptimal performance in visual understanding tasks. Conversely, directly modeling continuous semantic representations (e.g., CLIP, SigLIP) poses significant challenges in high-dimensional generative modeling, resulting in slow convergence and training instability. To resolve this dilemma, we introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous representation. We empirically demonstrate that reducing channel dimension is significantly more effective than spatial downsampling for both reconstruction and generation. Accordingly, we design an attention-based semantic compressor to distill dense features into a compact unified representation. Furthermore, we validate that the transfusion architecture surpasses query-based designs in convergence and consistency. Experiments demonstrate that UniCom achieves state-of-the-art generation performance among unified models. Notably, by preserving rich semantic priors, it delivers exceptional controllability in image editing and maintains image consistency even without relying on VAE.
69b28f13206bad330cd80141
Yaqi Zhao
admin_assigned
2026-03-12T10:40:26.130Z
65b5cc2516301f4030d35db7
/avatars/ba8d005c904ad3d8358fe69d518b51d7.svg
YaqiZhao
MiaTiancai
user
MiaTiancai
69b28f13206bad330cd80142
Wang Lin
69b28f13206bad330cd80143
Zijian Zhang
69b28f13206bad330cd80144
Miles Yang
69b28f13206bad330cd80145
Jingyuan Chen
69b28f13206bad330cd80146
Wentao Zhang
69b28f13206bad330cd80147
Zhao Zhong
69b28f13206bad330cd80148
Liefeng Bo
UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations
Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap. However, discretization inevitably discards fine-grained semantic information, leading to suboptimal performance in visual understanding tasks. Conversely, directly modeling continuous semantic representations (e.g., CLIP, SigLIP) poses significant challenges in high-dimensional generative modeling, resulting in slow convergence and training instability. To resolve this dilemma, we introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous representation. We empirically demonstrate that reducing channel dimension is significantly more effective than spatial downsampling for both reconstruction and generation. Accordingly, we design an attention-based semantic compressor to distill dense features into a compact unified representation. Furthermore, we validate that the transfusion architecture surpasses query-based designs in convergence and consistency. Experiments demonstrate that UniCom achieves state-of-the-art generation performance among unified models. Notably, by preserving rich semantic priors, it delivers exceptional controllability in image editing and maintains image consistency even without relying on VAE.
UniCom presents a unified multimodal framework that uses compressed continuous representations to improve visual understanding and generation while maintaining image consistency and controllability.
multimodal models visual tokenizers discrete visual tokenizers continuous semantic representations CLIP SigLIP high-dimensional generative modeling convergence training instability attention-based semantic compressor dense features compact unified representation transfusion architecture query-based designs image editing VAE