MetaCLIP 2: A Worldwide Scaling Recipe
Contrastive Language-Image Pretraining (CLIP) is a popular foundation model,
supporting from zero-shot classification, retrieval to encoders for multimodal
large language models (MLLMs). Although CLIP is successfully trained on
billion-scale image-text pairs from the English world, scaling CLIP's training
further to learning from the worldwide web data is still challenging: (1) no
curation method is available to handle data points from non-English world; (2)
the English performance from existing multilingual CLIP is worse than its
English-only counterpart, i.e., "curse of multilinguality" that is common in
LLMs. Here, we present MetaCLIP 2, the first recipe training CLIP from scratch
on worldwide web-scale image-text pairs. To generalize our findings, we conduct
rigorous ablations with minimal changes that are necessary to address the above
challenges and present a recipe enabling mutual benefits from English and
non-English world data. In zero-shot ImageNet classification, MetaCLIP 2
ViT-H/14 surpasses its English-only counterpart by 0.8% and mSigLIP by 0.7%,
and surprisingly sets new state-of-the-art without system-level confounding
factors (e.g., translation, bespoke architecture changes) on multilingual
benchmarks, such as CVQA with 57.4%, Babel-ImageNet with 50.2% and XM3600 with
64.3% on image-to-text retrieval.
688b6429292830d55fa68a2c
Yung-Sung Chuang
claimed_verified
2025-07-31T21:00:22.011Z
5df84571da6d0311fd3d5407
https://cdn-avatars.huggingface.co/v1/production/uploads/1650651305661-5df84571da6d0311fd3d5407.png
Yung-Sung Chuang
voidism
user
voidism
688b6429292830d55fa68a2d
Yang Li
claimed_verified
2025-08-06T19:31:55.479Z
672024b40a33ead6a3181fe0
/avatars/d77f6ba6d514940e3045328be78331eb.svg
Yang
yli1
user
yli1
688b6429292830d55fa68a2e
Dong Wang
claimed_verified
2025-08-06T19:31:51.198Z
640e26b03830fd441c2c106a
/avatars/8e457552ec454cbaa3ed9e138b29b47a.svg
Dong Wang
dongwang218
user
dongwang218
688b6429292830d55fa68a2f
Ching-Feng Yeh
688b6429292830d55fa68a30
Kehan Lyu
688b6429292830d55fa68a31
Ramya Raghavendra
688b6429292830d55fa68a32
James Glass
688b6429292830d55fa68a33
Lifei Huang
688b6429292830d55fa68a34
Jason Weston
688b6429292830d55fa68a35
Luke Zettlemoyer
688b6429292830d55fa68a36
Xinlei Chen
688b6429292830d55fa68a37
Zhuang Liu
688b6429292830d55fa68a38
Saining Xie
claimed_verified
2025-11-11T19:54:52.804Z
6596422646624a86ff3b3bda
/avatars/216e12b77e45ac5f1fa20932f5745411.svg
Saining Xie
sainx
user
sainx
688b6429292830d55fa68a39
Wen-tau Yih
688b6429292830d55fa68a3a
Shang-Wen Li
claimed_verified
2025-08-06T19:31:58.606Z
64e3a7b8e12618b261fff364
/avatars/860a5011f150537eb923f76d354f1bc5.svg
Shang-Wen Daniel Li
swdanielli
user
swdanielli
688b6429292830d55fa68a3b
Hu Xu
MetaCLIP 2: A Worldwide Scaling Recipe
Contrastive Language-Image Pretraining (CLIP) is a popular foundation model,
supporting from zero-shot classification, retrieval to encoders for multimodal
large language models (MLLMs). Although CLIP is successfully trained on
billion-scale image-text pairs from the English world, scaling CLIP's training
further to learning from the worldwide web data is still challenging: (1) no
curation method is available to handle data points from non-English world; (2)
the English performance from existing multilingual CLIP is worse than its
English-only counterpart, i.e., "curse of multilinguality" that is common in
LLMs. Here, we present MetaCLIP 2, the first recipe training CLIP from scratch
on worldwide web-scale image-text pairs. To generalize our findings, we conduct
rigorous ablations with minimal changes that are necessary to address the above
challenges and present a recipe enabling mutual benefits from English and
non-English world data. In zero-shot ImageNet classification, MetaCLIP 2
ViT-H/14 surpasses its English-only counterpart by 0.8% and mSigLIP by 0.7%,
and surprisingly sets new state-of-the-art without system-level confounding
factors (e.g., translation, bespoke architecture changes) on multilingual
benchmarks, such as CVQA with 57.4%, Babel-ImageNet with 50.2% and XM3600 with
64.3% on image-to-text retrieval.
MetaCLIP 2, trained on worldwide web-scale image-text pairs, improves zero-shot classification and multilingual benchmarks without system-level confounding factors.
Contrastive Language-Image Pretraining CLIP multilingual CLIP MetaCLIP 2 zero-shot classification image-text pairs multilingual benchmarks CVQA Babel-ImageNet XM3600 image-to-text retrieval