FireRed-OCR Technical Report
We present FireRed-OCR, a systematic framework to specialize general VLMs into high-performance OCR models. Large Vision-Language Models (VLMs) have demonstrated impressive general capabilities but frequently suffer from ``structural hallucination'' when processing complex documents, limiting their utility in industrial OCR applications. In this paper, we introduce FireRed-OCR, a novel framework designed to transform general-purpose VLMs (based on Qwen3-VL) into pixel-precise structural document parsing experts. To address the scarcity of high-quality structured data, we construct a ``Geometry + Semantics'' Data Factory. Unlike traditional random sampling, our pipeline leverages geometric feature clustering and multi-dimensional tagging to synthesize and curate a highly balanced dataset, effectively handling long-tail layouts and rare document types. Furthermore, we propose a Three-Stage Progressive Training strategy that guides the model from pixel-level perception to logical structure generation. This curriculum includes: (1) Multi-task Pre-alignment to ground the model's understanding of document structure; (2) Specialized SFT for standardizing full-image Markdown output; and (3) Format-Constrained Group Relative Policy Optimization (GRPO), which utilizes reinforcement learning to enforce strict syntactic validity and structural integrity (e.g., table closure, formula syntax). Extensive evaluations on OmniDocBench v1.5 demonstrate that FireRed-OCR achieves state-of-the-art performance with an overall score of 92.94\%, significantly outperforming strong baselines such as DeepSeek-OCR 2 and OCRVerse across text, formula, table, and reading order metrics. We open-source our code and model weights to facilitate the ``General VLM to Specialized Structural Expert'' paradigm.
69a666bb663a122a7ce6676f
Hao Wu
69a666bb663a122a7ce66770
Haoran Lou
69a666bb663a122a7ce66771
Xinyue Li
69a666bb663a122a7ce66772
Zuodong Zhong
claimed_verified
2026-03-06T13:57:18.778Z
67b73d970d11464dcbc9f49b
/avatars/6bc43f408ba75b87bba29d8cf87a5ae9.svg
钟祚栋
Zzdpaaa
user
Zzdpaaa
69a666bb663a122a7ce66773
Zhaojun Sun
69a666bb663a122a7ce66774
Phellon Chen
69a666bb663a122a7ce66775
Xuanhe Zhou
69a666bb663a122a7ce66776
Kai Zuo
69a666bb663a122a7ce66777
Yibo Chen
69a666bb663a122a7ce66778
Xu Tang
69a666bb663a122a7ce66779
Yao Hu
69a666bb663a122a7ce6677a
Boxiang Zhou
69a666bb663a122a7ce6677b
Jian Wu
69a666bb663a122a7ce6677c
Yongji Wu
69a666bb663a122a7ce6677d
Wenxin Yu
69a666bb663a122a7ce6677e
Yingmiao Liu
69a666bb663a122a7ce6677f
Yuhao Huang
69a666bb663a122a7ce66780
Manjie Xu
69a666bb663a122a7ce66781
Gang Liu
69a666bb663a122a7ce66782
Yidong Ma
69a666bb663a122a7ce66783
Zhichao Sun
69a666bb663a122a7ce66784
Changhao Qiao
FireRed-OCR Technical Report
We present FireRed-OCR, a systematic framework to specialize general VLMs into high-performance OCR models. Large Vision-Language Models (VLMs) have demonstrated impressive general capabilities but frequently suffer from ``structural hallucination'' when processing complex documents, limiting their utility in industrial OCR applications. In this paper, we introduce FireRed-OCR, a novel framework designed to transform general-purpose VLMs (based on Qwen3-VL) into pixel-precise structural document parsing experts. To address the scarcity of high-quality structured data, we construct a ``Geometry + Semantics'' Data Factory. Unlike traditional random sampling, our pipeline leverages geometric feature clustering and multi-dimensional tagging to synthesize and curate a highly balanced dataset, effectively handling long-tail layouts and rare document types. Furthermore, we propose a Three-Stage Progressive Training strategy that guides the model from pixel-level perception to logical structure generation. This curriculum includes: (1) Multi-task Pre-alignment to ground the model's understanding of document structure; (2) Specialized SFT for standardizing full-image Markdown output; and (3) Format-Constrained Group Relative Policy Optimization (GRPO), which utilizes reinforcement learning to enforce strict syntactic validity and structural integrity (e.g., table closure, formula syntax). Extensive evaluations on OmniDocBench v1.5 demonstrate that FireRed-OCR achieves state-of-the-art performance with an overall score of 92.94\%, significantly outperforming strong baselines such as DeepSeek-OCR 2 and OCRVerse across text, formula, table, and reading order metrics. We open-source our code and model weights to facilitate the ``General VLM to Specialized Structural Expert'' paradigm.
FireRed-OCR transforms general vision-language models into specialized OCR systems through structured data synthesis and progressive training strategies.
Vision-Language Models structural hallucination document parsing Geometry + Semantics Data Factory geometric feature clustering multi-dimensional tagging Three-Stage Progressive Training Multi-task Pre-alignment Specialized SFT Format-Constrained Group Relative Policy Optimization GRPO reinforcement learning syntactic validity structural integrity OmniDocBench