DiscoX: Benchmarking Discourse-Level Translation task in Expert Domains
The evaluation of discourse-level translation in expert domains remains inadequate, despite its centrality to knowledge dissemination and cross-lingual scholarly communication. While these translations demand discourse-level coherence and strict terminological precision, current evaluation methods predominantly focus on segment-level accuracy and fluency. To address this limitation, we introduce DiscoX, a new benchmark for discourse-level and expert-level Chinese-English translation. It comprises 200 professionally-curated texts from 7 domains, with an average length exceeding 1700 tokens. To evaluate performance on DiscoX, we also develop Metric-S, a reference-free system that provides fine-grained automatic assessments across accuracy, fluency, and appropriateness. Metric-S demonstrates strong consistency with human judgments, significantly outperforming existing metrics. Our experiments reveal a remarkable performance gap: even the most advanced LLMs still trail human experts on these tasks. This finding validates the difficulty of DiscoX and underscores the challenges that remain in achieving professional-grade machine translation. The proposed benchmark and evaluation system provide a robust framework for more rigorous evaluation, facilitating future advancements in LLM-based translation.
691a98606bfd5965c0fd349e
Xiying Zhao
691a98606bfd5965c0fd349f
Zhoufutu Wen
691a98606bfd5965c0fd34a0
Zhixuan Chen
691a98606bfd5965c0fd34a1
Jingzhe Ding
claimed_verified
2025-12-16T10:18:30.949Z
6737eb499ab5a7623aaf038a
/avatars/dfe21c7db767a34f2aff3e80cebcfa22.svg
Jingzhe Ding
JingzheDing
user
JingzheDing
691a98606bfd5965c0fd34a2
Jianpeng Jiao
691a98606bfd5965c0fd34a3
Shuai Li
691a98606bfd5965c0fd34a4
Xi Li
691a98606bfd5965c0fd34a5
Danni Liang
691a98606bfd5965c0fd34a6
Shengda Long
691a98606bfd5965c0fd34a7
Qianqian Liu
691a98606bfd5965c0fd34a8
Xianbo Wu
691a98606bfd5965c0fd34a9
Hongwan Gao
691a98606bfd5965c0fd34aa
Xiang Gao
claimed_verified
2025-12-16T09:49:13.212Z
671068b8243baf4c58fc29c2
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/leMkq-kw1icPo2bCLivGv.png
Xiang Gao
coffiney
user
coffiney
691a98606bfd5965c0fd34ab
Liang Hu
691a98606bfd5965c0fd34ac
Jiashuo Liu
691a98606bfd5965c0fd34ad
Mengyun Liu
691a98606bfd5965c0fd34ae
Weiran Shi
691a98606bfd5965c0fd34af
Chenghao Yang
691a98606bfd5965c0fd34b0
Qianyu Yang
691a98606bfd5965c0fd34b1
Xuanliang Zhang
691a98606bfd5965c0fd34b2
Ge Zhang
claimed_verified
2025-11-18T14:09:14.126Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
zhangysk
user
zhangysk
691a98606bfd5965c0fd34b3
Wenhao Huang
DiscoX: Benchmarking Discourse-Level Translation task in Expert Domains
The evaluation of discourse-level translation in expert domains remains inadequate, despite its centrality to knowledge dissemination and cross-lingual scholarly communication. While these translations demand discourse-level coherence and strict terminological precision, current evaluation methods predominantly focus on segment-level accuracy and fluency. To address this limitation, we introduce DiscoX, a new benchmark for discourse-level and expert-level Chinese-English translation. It comprises 200 professionally-curated texts from 7 domains, with an average length exceeding 1700 tokens. To evaluate performance on DiscoX, we also develop Metric-S, a reference-free system that provides fine-grained automatic assessments across accuracy, fluency, and appropriateness. Metric-S demonstrates strong consistency with human judgments, significantly outperforming existing metrics. Our experiments reveal a remarkable performance gap: even the most advanced LLMs still trail human experts on these tasks. This finding validates the difficulty of DiscoX and underscores the challenges that remain in achieving professional-grade machine translation. The proposed benchmark and evaluation system provide a robust framework for more rigorous evaluation, facilitating future advancements in LLM-based translation.
A new benchmark DiscoX and evaluation system Metric-S are introduced to assess discourse-level and expert-level Chinese-English translation, highlighting the challenges in achieving professional-grade machine translation.
discourse-level translation DiscoX Metric-S reference-free system accuracy fluency appropriateness human judgments LLMs professional-grade machine translation