GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cluster of high- and mid-resource scripts. We introduce GlotOCR Bench, a comprehensive benchmark evaluating OCR generalization across 100+ Unicode scripts. Our benchmark comprises clean and degraded image variants rendered from real multilingual texts. Images are rendered using fonts from the Google Fonts repository, shaped with HarfBuzz and rasterized with FreeType, supporting both LTR and RTL scripts. Samples of rendered images were manually reviewed to verify correct rendering across all scripts. We evaluate a broad suite of open-weight and proprietary vision-language models and find that most perform well on fewer than ten scripts, and even the strongest frontier models fail to generalize beyond thirty scripts. Performance broadly tracks script-level pretraining coverage, suggesting that current OCR systems rely on language model pretraining as much as on visual recognition. Models confronted with unfamiliar scripts either produce random noise or hallucinate characters from similar scripts they already know. We release the benchmark and pipeline for reproducibility. Pipeline Code: https://github.com/cisnlp/glotocr-bench, Benchmark: https://hf.co/datasets/cis-lmu/glotocr-bench.
69dfa2b0f93805b185f6dba5
Amir Hossein Kargaran
claimed_verified
2026-04-16T08:02:50.949Z
61bf84c8ca59d6d196a1b4e8
https://cdn-avatars.huggingface.co/v1/production/uploads/61bf84c8ca59d6d196a1b4e8/5CTo8gpBXKeKA5cbQLU7b.jpeg
Amir Hossein Kargaran
kargaranamir
user
kargaranamir
69dfa2b0f93805b185f6dba6
Nafiseh Nikeghbal
claimed_verified
2026-04-16T08:02:48.832Z
62cb464b6193ba3ced7e5082
https://cdn-avatars.huggingface.co/v1/production/uploads/62cb464b6193ba3ced7e5082/CBovM4H7ESy3UuHk2etW9.jpeg
nikeghbal
nafisehNik
user
nafisehNik
69dfa2b0f93805b185f6dba7
Jana Diesner
69dfa2b0f93805b185f6dba8
François Yvon
69dfa2b0f93805b185f6dba9
Hinrich Schütze
GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cluster of high- and mid-resource scripts. We introduce GlotOCR Bench, a comprehensive benchmark evaluating OCR generalization across 100+ Unicode scripts. Our benchmark comprises clean and degraded image variants rendered from real multilingual texts. Images are rendered using fonts from the Google Fonts repository, shaped with HarfBuzz and rasterized with FreeType, supporting both LTR and RTL scripts. Samples of rendered images were manually reviewed to verify correct rendering across all scripts. We evaluate a broad suite of open-weight and proprietary vision-language models and find that most perform well on fewer than ten scripts, and even the strongest frontier models fail to generalize beyond thirty scripts. Performance broadly tracks script-level pretraining coverage, suggesting that current OCR systems rely on language model pretraining as much as on visual recognition. Models confronted with unfamiliar scripts either produce random noise or hallucinate characters from similar scripts they already know. We release the benchmark and pipeline for reproducibility. Pipeline Code: https://github.com/cisnlp/glotocr-bench, Benchmark: https://hf.co/datasets/cis-lmu/glotocr-bench.
Vision-language models show limited generalization in OCR across diverse scripts, with performance closely tied to pretraining coverage and struggling with unfamiliar writing systems.
optical character recognition vision-language models Unicode scripts multilingual texts HarfBuzz FreeType LTR RTL pretraining coverage