UniWeTok: An Unified Binary Tokenizer with Codebook Size 2^{128} for Unified Multimodal Large Language Model
Unified Multimodal Large Language Models (MLLMs) require a visual representation that simultaneously supports high-fidelity reconstruction, complex semantic extraction, and generative suitability. However, existing visual tokenizers typically struggle to satisfy these conflicting objectives within a single framework. In this paper, we introduce UniWeTok, a unified discrete tokenizer designed to bridge this gap using a massive binary codebook (2^{128}). For training framework, we introduce Pre-Post Distillation and a Generative-Aware Prior to enhance the semantic extraction and generative prior of the discrete tokens. In terms of model architecture, we propose a convolution-attention hybrid architecture with the SigLu activation function. SigLu activation not only bounds the encoder output and stabilizes the semantic distillation process but also effectively addresses the optimization conflict between token entropy loss and commitment loss. We further propose a three-stage training framework designed to enhance UniWeTok's adaptability cross various image resolutions and perception-sensitive scenarios, such as those involving human faces and textual content. On ImageNet, UniWeTok achieves state-of-the-art image generation performance (FID: UniWeTok 1.38 vs. REPA 1.42) while requiring a remarkably low training compute (Training Tokens: UniWeTok 33B vs. REPA 262B). On general-domain, UniWeTok demonstrates highly competitive capabilities across a broad range of tasks, including multimodal understanding, image generation (DPG Score: UniWeTok 86.63 vs. FLUX.1 [Dev] 83.84), and editing (GEdit Overall Score: UniWeTok 5.09 vs. OmniGen 5.06). We release code and models to facilitate community exploration of unified tokenizer and MLLM.
6993e66350fb2c0be4783d6b
Shaobin Zhuang
6993e66350fb2c0be4783d6c
Yuang Ai
claimed_verified
2026-02-18T09:07:05.519Z
65709464b3501cbcb8f4007e
https://cdn-avatars.huggingface.co/v1/production/uploads/65709464b3501cbcb8f4007e/lD6_mJvhiGPqYxIXhkKal.jpeg
Yuang Ai
shallowdream204
user
shallowdream204
6993e66350fb2c0be4783d6d
Jiaming Han
claimed_verified
2026-02-18T09:07:03.664Z
62318c0386753f5f41d0e261
https://cdn-avatars.huggingface.co/v1/production/uploads/62318c0386753f5f41d0e261/xO_5PvOf7lXhQPnQLcmnq.jpeg
Jiaming Han
csuhan
user
csuhan
6993e66350fb2c0be4783d6e
Weijia Mao
6993e66350fb2c0be4783d6f
Xiaohui Li
6993e66350fb2c0be4783d70
Fangyikang Wang
6993e66350fb2c0be4783d71
Xiao Wang
6993e66350fb2c0be4783d72
Yan Li
6993e66350fb2c0be4783d73
Shanchuan Lin
6993e66350fb2c0be4783d74
Kun Xu
6993e66350fb2c0be4783d75
Zhenheng Yang
6993e66350fb2c0be4783d76
Huaibo Huang
6993e66350fb2c0be4783d77
Xiangyu Yue
6993e66350fb2c0be4783d78
Hao Chen
6993e66350fb2c0be4783d79
Yali Wang
UniWeTok: An Unified Binary Tokenizer with Codebook Size 2^{128} for Unified Multimodal Large Language Model
Unified Multimodal Large Language Models (MLLMs) require a visual representation that simultaneously supports high-fidelity reconstruction, complex semantic extraction, and generative suitability. However, existing visual tokenizers typically struggle to satisfy these conflicting objectives within a single framework. In this paper, we introduce UniWeTok, a unified discrete tokenizer designed to bridge this gap using a massive binary codebook (2^{128}). For training framework, we introduce Pre-Post Distillation and a Generative-Aware Prior to enhance the semantic extraction and generative prior of the discrete tokens. In terms of model architecture, we propose a convolution-attention hybrid architecture with the SigLu activation function. SigLu activation not only bounds the encoder output and stabilizes the semantic distillation process but also effectively addresses the optimization conflict between token entropy loss and commitment loss. We further propose a three-stage training framework designed to enhance UniWeTok's adaptability cross various image resolutions and perception-sensitive scenarios, such as those involving human faces and textual content. On ImageNet, UniWeTok achieves state-of-the-art image generation performance (FID: UniWeTok 1.38 vs. REPA 1.42) while requiring a remarkably low training compute (Training Tokens: UniWeTok 33B vs. REPA 262B). On general-domain, UniWeTok demonstrates highly competitive capabilities across a broad range of tasks, including multimodal understanding, image generation (DPG Score: UniWeTok 86.63 vs. FLUX.1 [Dev] 83.84), and editing (GEdit Overall Score: UniWeTok 5.09 vs. OmniGen 5.06). We release code and models to facilitate community exploration of unified tokenizer and MLLM.
UniWeTok introduces a unified discrete tokenizer with a massive binary codebook and novel training techniques to achieve superior performance in image generation and multimodal tasks while reducing computational requirements.
visual tokenizers discrete tokenizer binary codebook Pre-Post Distillation Generative-Aware Prior convolution-attention hybrid architecture SigLu activation function token entropy loss commitment loss three-stage training framework multimodal understanding image generation DPG Score GEdit Overall Score