OPV: Outcome-based Process Verifier for Efficient Long Chain-of-Thought Verification
Large language models (LLMs) have achieved significant progress in solving complex reasoning tasks by Reinforcement Learning with Verifiable Rewards (RLVR). This advancement is also inseparable from the oversight automated by reliable verifiers. However, current outcome-based verifiers (OVs) are unable to inspect the unreliable intermediate steps in the long reasoning chains of thought (CoTs). Meanwhile, current process-based verifiers (PVs) have difficulties in reliably detecting errors in the complex long CoTs, limited by the scarcity of high-quality annotations due to the prohibitive costs of human annotations. Therefore, we propose the Outcome-based Process Verifier (OPV), which verifies the rationale process of summarized outcomes from long CoTs to achieve both accurate and efficient verification and enable large-scale annotation. To empower the proposed verifier, we adopt an iterative active learning framework with expert annotations to progressively improve the verification capability of OPV with fewer annotation costs. Specifically, in each iteration, the most uncertain cases of the current best OPV are annotated and then subsequently used to train a new OPV through Rejection Fine-Tuning (RFT) and RLVR for the next round. Extensive experiments demonstrate OPV's superior performance and broad applicability. It achieves new state-of-the-art results on our held-out OPV-Bench, outperforming much larger open-source models such as Qwen3-Max-Preview with an F1 score of 83.1 compared to 76.3. Furthermore, OPV effectively detects false positives within synthetic dataset, closely align with expert assessment. When collaborating with policy models, OPV consistently yields performance gains, e.g., raising the accuracy of DeepSeek-R1-Distill-Qwen-32B from 55.2% to 73.3% on AIME2025 as the compute budget scales.
693b91539874a2a5e4ffb318
Zijian Wu
693b91539874a2a5e4ffb319
Lingkai Kong
693b91539874a2a5e4ffb31a
Wenwei Zhang
693b91539874a2a5e4ffb31b
Songyang Gao
693b91539874a2a5e4ffb31c
Yuzhe Gu
claimed_verified
2025-12-12T09:15:02.946Z
6601196cc91ba4c08ad6e270
https://cdn-avatars.huggingface.co/v1/production/uploads/6601196cc91ba4c08ad6e270/venywO3WPi2fNi5WUJTH0.jpeg
Yuzhe Gu
vanilla1116
user
vanilla1116
693b91539874a2a5e4ffb31d
Zhongrui Cai
693b91539874a2a5e4ffb31e
Tianyou Ma
693b91539874a2a5e4ffb31f
Yuhong Liu
693b91539874a2a5e4ffb320
Zhi Wang
693b91539874a2a5e4ffb321
Runyuan Ma
693b91539874a2a5e4ffb322
Guangyu Wang
693b91539874a2a5e4ffb323
Wei Li
693b91539874a2a5e4ffb324
Conghui He
693b91539874a2a5e4ffb325
Dahua Lin
693b91539874a2a5e4ffb326
Kai Chen
OPV: Outcome-based Process Verifier for Efficient Long Chain-of-Thought Verification
Large language models (LLMs) have achieved significant progress in solving complex reasoning tasks by Reinforcement Learning with Verifiable Rewards (RLVR). This advancement is also inseparable from the oversight automated by reliable verifiers. However, current outcome-based verifiers (OVs) are unable to inspect the unreliable intermediate steps in the long reasoning chains of thought (CoTs). Meanwhile, current process-based verifiers (PVs) have difficulties in reliably detecting errors in the complex long CoTs, limited by the scarcity of high-quality annotations due to the prohibitive costs of human annotations. Therefore, we propose the Outcome-based Process Verifier (OPV), which verifies the rationale process of summarized outcomes from long CoTs to achieve both accurate and efficient verification and enable large-scale annotation. To empower the proposed verifier, we adopt an iterative active learning framework with expert annotations to progressively improve the verification capability of OPV with fewer annotation costs. Specifically, in each iteration, the most uncertain cases of the current best OPV are annotated and then subsequently used to train a new OPV through Rejection Fine-Tuning (RFT) and RLVR for the next round. Extensive experiments demonstrate OPV's superior performance and broad applicability. It achieves new state-of-the-art results on our held-out OPV-Bench, outperforming much larger open-source models such as Qwen3-Max-Preview with an F1 score of 83.1 compared to 76.3. Furthermore, OPV effectively detects false positives within synthetic dataset, closely align with expert assessment. When collaborating with policy models, OPV consistently yields performance gains, e.g., raising the accuracy of DeepSeek-R1-Distill-Qwen-32B from 55.2% to 73.3% on AIME2025 as the compute budget scales.
The Outcome-based Process Verifier (OPV) improves the verification of complex reasoning chains in large language models by combining outcome-based and process-based verification with iterative active learning and Rejection Fine-Tuning, achieving state-of-the-art performance on various benchmarks.
Reinforcement Learning with Verifiable Rewards (RLVR) verifiers outcome-based verifiers (OVs) process-based verifiers (PVs) CoTs iterative active learning Rejection Fine-Tuning (RFT) OPV-Bench AIME2025 DeepSeek-R1-Distill-Qwen-32B