MR-Align: Meta-Reasoning Informed Factuality Alignment for Large
  Reasoning Models
Large reasoning models (LRMs) show strong capabilities in complex reasoning,
yet their marginal gains on evidence-dependent factual questions are limited.
We find this limitation is partially attributable to a reasoning-answer hit
gap, where the model identifies the correct facts during reasoning but fails to
incorporate them into the final response, thereby reducing factual fidelity. To
address this issue, we propose MR-ALIGN, a Meta-Reasoning informed alignment
framework that enhances factuality without relying on external verifiers.
MR-ALIGN quantifies state transition probabilities along the model's thinking
process and constructs a transition-aware implicit reward that reinforces
beneficial reasoning patterns while suppressing defective ones at the atomic
thinking segments. This re-weighting reshapes token-level signals into
probability-aware segment scores, encouraging coherent reasoning trajectories
that are more conducive to factual correctness. Empirical evaluations across
four factual QA datasets and one long-form factuality benchmark show that
MR-ALIGN consistently improves accuracy and truthfulness while reducing
misleading reasoning. These results highlight that aligning the reasoning
process itself, rather than merely the outputs, is pivotal for advancing
factuality in LRMs.
69081d1b812eca10f9cc5e6c
Xinming Wang
claimed_verified
2025-11-03T20:52:19.280Z
66e4221d99e55519e9be00b6
/avatars/aa821e632ce314cb636c3a1469d61017.svg
Wang
PeterKKQ
user
PeterKKQ
69081d1b812eca10f9cc5e6d
Jian Xu
69081d1b812eca10f9cc5e6e
Bin Yu
69081d1b812eca10f9cc5e6f
Sheng Lian
69081d1b812eca10f9cc5e70
Hongzhu Yi
claimed_verified
2025-11-04T20:27:09.213Z
641eec9973cfc036ddb9a22e
/avatars/b17b0293bc47ffc65585732e32db66f8.svg
易泓竹
hongzhuyi
user
hongzhuyi
69081d1b812eca10f9cc5e71
Yi Chen
claimed_verified
2025-11-05T16:25:13.053Z
6729bbdebd5ca35b8ed8cc1c
https://cdn-avatars.huggingface.co/v1/production/uploads/6729bbdebd5ca35b8ed8cc1c/FaklBmA2UqJrxuF2balW0.jpeg
Yi Chen
Banjiuyufen
user
Banjiuyufen
69081d1b812eca10f9cc5e72
Yingjian Zhu
69081d1b812eca10f9cc5e73
Boran Wang
69081d1b812eca10f9cc5e74
Hongming Yang
69081d1b812eca10f9cc5e75
Han Hu
69081d1b812eca10f9cc5e76
Xu-Yao Zhang
69081d1b812eca10f9cc5e77
Cheng-Lin Liu
MR-Align: Meta-Reasoning Informed Factuality Alignment for Large
  Reasoning Models
Large reasoning models (LRMs) show strong capabilities in complex reasoning,
yet their marginal gains on evidence-dependent factual questions are limited.
We find this limitation is partially attributable to a reasoning-answer hit
gap, where the model identifies the correct facts during reasoning but fails to
incorporate them into the final response, thereby reducing factual fidelity. To
address this issue, we propose MR-ALIGN, a Meta-Reasoning informed alignment
framework that enhances factuality without relying on external verifiers.
MR-ALIGN quantifies state transition probabilities along the model's thinking
process and constructs a transition-aware implicit reward that reinforces
beneficial reasoning patterns while suppressing defective ones at the atomic
thinking segments. This re-weighting reshapes token-level signals into
probability-aware segment scores, encouraging coherent reasoning trajectories
that are more conducive to factual correctness. Empirical evaluations across
four factual QA datasets and one long-form factuality benchmark show that
MR-ALIGN consistently improves accuracy and truthfulness while reducing
misleading reasoning. These results highlight that aligning the reasoning
process itself, rather than merely the outputs, is pivotal for advancing
factuality in LRMs.
MR-ALIGN, a Meta-Reasoning informed alignment framework, enhances the factuality of large reasoning models by aligning their reasoning process, improving accuracy and reducing misleading reasoning.
Large reasoning models reasoning-answer hit gap Meta-Reasoning MR-ALIGN state transition probabilities transition-aware implicit reward token-level signals probability-aware segment scores coherent reasoning trajectories factual QA datasets long-form factuality benchmark