TRivia: Self-supervised Fine-tuning of Vision-Language Models for Table Recognition
Table recognition (TR) aims to transform table images into semi-structured representations such as HTML or Markdown. As a core component of document parsing, TR has long relied on supervised learning, with recent efforts dominated by fine-tuning vision-language models (VLMs) using labeled data. While VLMs have brought TR to the next level, pushing performance further demands large-scale labeled data that is costly to obtain. Consequently, although proprietary models have continuously pushed the performance boundary, open-source models, often trained with limited resources and, in practice, the only viable option for many due to privacy regulations, still lag far behind. To bridge this gap, we introduce TRivia, a self-supervised fine-tuning method that enables pretrained VLMs to learn TR directly from unlabeled table images in the wild. Built upon Group Relative Policy Optimization, TRivia automatically identifies unlabeled samples that most effectively facilitate learning and eliminates the need for human annotations through a question-answering-based reward mechanism. An attention-guided module generates diverse questions for each table image, and the ability to interpret the recognition results and answer them correctly provides feedback to optimize the TR model. This closed-loop process allows the TR model to autonomously learn to recognize, structure, and reason over tables without labeled data. Leveraging this pipeline, we present TRivia-3B, an open-sourced, compact, and state-of-the-art TR model that surpasses existing systems (e.g., Gemini 2.5 Pro, MinerU2.5) on three popular benchmarks. Model and code are released at: https://github.com/opendatalab/TRivia
692e868f37312eaa83fd8910
Junyuan Zhang
claimed_verified
2026-03-30T09:12:41.110Z
65411de281a8731a1c28293d
/avatars/18607dcd65303e1f688fae8015f43314.svg
junyuan
Carkham
user
Carkham
692e868f37312eaa83fd8911
Bin Wang
claimed_verified
2026-04-07T08:50:00.039Z
63ae9ff5557befe297a76f90
https://cdn-avatars.huggingface.co/v1/production/uploads/1672388558183-noauth.jpeg
Bin Wang
wanderkid
user
wanderkid
692e868f37312eaa83fd8912
Qintong Zhang
692e868f37312eaa83fd8913
Fan Wu
692e868f37312eaa83fd8914
Zichen Wen
claimed_verified
2026-02-03T10:10:00.460Z
653b8c3e97a4d71d950e2f20
/avatars/b68880022e14556d0be58c69615db3be.svg
Zichen Wen
zichenwen
user
zichenwen
692e868f37312eaa83fd8915
Jialin Lu
692e868f37312eaa83fd8916
Junjie Shan
692e868f37312eaa83fd8917
Ziqi Zhao
692e868f37312eaa83fd8918
Shuya Yang
692e868f37312eaa83fd8919
Ziling Wang
692e868f37312eaa83fd891a
Ziyang Miao
692e868f37312eaa83fd891b
Huaping Zhong
692e868f37312eaa83fd891c
Yuhang Zang
claimed_verified
2026-03-13T08:15:49.316Z
63859cf3b2906edaf83af9f0
https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/kajwuVzd4pDucSPlwghxo.png
Yuhang Zang
yuhangzang
user
yuhangzang
692e868f37312eaa83fd891d
Xiaoyi Dong
692e868f37312eaa83fd891e
Ka-Ho Chow
692e868f37312eaa83fd891f
Conghui He
TRivia: Self-supervised Fine-tuning of Vision-Language Models for Table Recognition
Table recognition (TR) aims to transform table images into semi-structured representations such as HTML or Markdown. As a core component of document parsing, TR has long relied on supervised learning, with recent efforts dominated by fine-tuning vision-language models (VLMs) using labeled data. While VLMs have brought TR to the next level, pushing performance further demands large-scale labeled data that is costly to obtain. Consequently, although proprietary models have continuously pushed the performance boundary, open-source models, often trained with limited resources and, in practice, the only viable option for many due to privacy regulations, still lag far behind. To bridge this gap, we introduce TRivia, a self-supervised fine-tuning method that enables pretrained VLMs to learn TR directly from unlabeled table images in the wild. Built upon Group Relative Policy Optimization, TRivia automatically identifies unlabeled samples that most effectively facilitate learning and eliminates the need for human annotations through a question-answering-based reward mechanism. An attention-guided module generates diverse questions for each table image, and the ability to interpret the recognition results and answer them correctly provides feedback to optimize the TR model. This closed-loop process allows the TR model to autonomously learn to recognize, structure, and reason over tables without labeled data. Leveraging this pipeline, we present TRivia-3B, an open-sourced, compact, and state-of-the-art TR model that surpasses existing systems (e.g., Gemini 2.5 Pro, MinerU2.5) on three popular benchmarks. Model and code are released at: https://github.com/opendatalab/TRivia
TRivia, a self-supervised fine-tuning method, enables pretrained vision-language models to learn table recognition from unlabeled data using a question-answering-based reward mechanism, outperforming existing models on popular benchmarks.
self-supervised fine-tuning vision-language models Group Relative Policy Optimization attention-guided module question-answering-based reward mechanism table recognition TRivia TRivia-3B Gemini 2.5 Pro MinerU2.5