M3DR: Towards Universal Multilingual Multimodal Document Retrieval
Multimodal document retrieval systems have shown strong progress in aligning visual and textual content for semantic search. However, most existing approaches remain heavily English-centric, limiting their effectiveness in multilingual contexts. In this work, we present M3DR (Multilingual Multimodal Document Retrieval), a framework designed to bridge this gap across languages, enabling applicability across diverse linguistic and cultural contexts. M3DR leverages synthetic multilingual document data and generalizes across different vision-language architectures and model sizes, enabling robust cross-lingual and cross-modal alignment. Using contrastive training, our models learn unified representations for text and document images that transfer effectively across languages. We validate this capability on 22 typologically diverse languages, demonstrating consistent performance and adaptability across linguistic and script variations. We further introduce a comprehensive benchmark that captures real-world multilingual scenarios, evaluating models under monolingual, multilingual, and mixed-language settings. M3DR generalizes across both single dense vector and ColBERT-style token-level multi-vector retrieval paradigms. Our models, NetraEmbed and ColNetraEmbed achieve state-of-the-art performance with ~150% relative improvements on cross-lingual retrieval.
69311bb22d1e5b0a7d84da33
Adithya S Kolavi
claimed_verified
2025-12-08T08:32:49.573Z
6442d975ad54813badc1ddf7
https://cdn-avatars.huggingface.co/v1/production/uploads/6442d975ad54813badc1ddf7/JIAN8B1dLOXN8fWTk8_mI.jpeg
Adithya S K
AdithyaSK
user
AdithyaSK
69311bb22d1e5b0a7d84da34
Vyoman Jain
claimed_verified
2025-12-08T08:32:54.078Z
665d52766c20caf090adc94e
https://cdn-avatars.huggingface.co/v1/production/uploads/665d52766c20caf090adc94e/-xnnLMCI6HCBfvhny7MgP.jpeg
VyoJ
VyoJ
user
VyoJ
M3DR: Towards Universal Multilingual Multimodal Document Retrieval
Multimodal document retrieval systems have shown strong progress in aligning visual and textual content for semantic search. However, most existing approaches remain heavily English-centric, limiting their effectiveness in multilingual contexts. In this work, we present M3DR (Multilingual Multimodal Document Retrieval), a framework designed to bridge this gap across languages, enabling applicability across diverse linguistic and cultural contexts. M3DR leverages synthetic multilingual document data and generalizes across different vision-language architectures and model sizes, enabling robust cross-lingual and cross-modal alignment. Using contrastive training, our models learn unified representations for text and document images that transfer effectively across languages. We validate this capability on 22 typologically diverse languages, demonstrating consistent performance and adaptability across linguistic and script variations. We further introduce a comprehensive benchmark that captures real-world multilingual scenarios, evaluating models under monolingual, multilingual, and mixed-language settings. M3DR generalizes across both single dense vector and ColBERT-style token-level multi-vector retrieval paradigms. Our models, NetraEmbed and ColNetraEmbed achieve state-of-the-art performance with ~150% relative improvements on cross-lingual retrieval.
M3DR is a multilingual multimodal document retrieval framework using contrastive training to achieve robust cross-lingual and cross-modal alignment across diverse languages and document types.
M3DR synthetic multilingual document data vision-language architectures contrastive training unified representations ColBERT-style token-level multi-vector NetraEmbed ColNetraEmbed