V_1: Unifying Generation and Self-Verification for Parallel Reasoners
Test-time scaling for complex reasoning tasks shows that leveraging inference-time compute, by methods such as independently sampling and aggregating multiple solutions, results in significantly better task outcomes. However, a critical bottleneck is verification: sampling is only effective if correct solutions can be reliably identified among candidates. While existing approaches typically evaluate candidates independently via scalar scoring, we demonstrate that models are substantially stronger at pairwise self-verification. Leveraging this insight, we introduce V_1, a framework that unifies generation and verification through efficient pairwise ranking. V_1 comprises two components: V_1-Infer, an uncertainty-guided algorithm using a tournament-based ranking that dynamically allocates self-verification compute to candidate pairs whose relative correctness is most uncertain; and V_1-PairRL, an RL framework that jointly trains a single model as both generator and pairwise self-verifier, ensuring the verifier adapts to the generator's evolving distribution. On code generation (LiveCodeBench, CodeContests, SWE-Bench) and math reasoning (AIME, HMMT) benchmarks, V_1-Infer improves Pass@1 by up to 10% over pointwise verification and outperforms recent test-time scaling methods while being significantly more efficient. Furthermore, V_1-PairRL achieves 7--9% test-time scaling gains over standard RL and pointwise joint training, and improves base Pass@1 by up to 8.7% over standard RL in a code-generation setting.
69a9a816c4f6edddcc6c8512
Harman Singh
claimed_verified
2026-04-07T08:48:40.756Z
631c4048e65cd7f686d2ad4f
/avatars/49902bcd18da8079bacf312edc386982.svg
Harman Singh
harman
user
harman
69a9a816c4f6edddcc6c8513
Xiuyu Li
69a9a816c4f6edddcc6c8514
Kusha Sareen
69a9a816c4f6edddcc6c8515
Monishwaran Maheswaran
69a9a816c4f6edddcc6c8516
Sijun Tan
69a9a816c4f6edddcc6c8517
Xiaoxia Wu
69a9a816c4f6edddcc6c8518
Junxiong Wang
69a9a816c4f6edddcc6c8519
Alpay Ariyak
69a9a816c4f6edddcc6c851a
Qingyang Wu
69a9a816c4f6edddcc6c851b
Samir Khaki
69a9a816c4f6edddcc6c851c
Rishabh Tiwari
69a9a816c4f6edddcc6c851d
Long Lian
69a9a816c4f6edddcc6c851e
Yucheng Lu
69a9a816c4f6edddcc6c851f
Boyi Li
69a9a816c4f6edddcc6c8520
Alane Suhr
69a9a816c4f6edddcc6c8521
Ben Athiwaratkun
69a9a816c4f6edddcc6c8522
Kurt Keutzer
V_1: Unifying Generation and Self-Verification for Parallel Reasoners
Test-time scaling for complex reasoning tasks shows that leveraging inference-time compute, by methods such as independently sampling and aggregating multiple solutions, results in significantly better task outcomes. However, a critical bottleneck is verification: sampling is only effective if correct solutions can be reliably identified among candidates. While existing approaches typically evaluate candidates independently via scalar scoring, we demonstrate that models are substantially stronger at pairwise self-verification. Leveraging this insight, we introduce V_1, a framework that unifies generation and verification through efficient pairwise ranking. V_1 comprises two components: V_1-Infer, an uncertainty-guided algorithm using a tournament-based ranking that dynamically allocates self-verification compute to candidate pairs whose relative correctness is most uncertain; and V_1-PairRL, an RL framework that jointly trains a single model as both generator and pairwise self-verifier, ensuring the verifier adapts to the generator's evolving distribution. On code generation (LiveCodeBench, CodeContests, SWE-Bench) and math reasoning (AIME, HMMT) benchmarks, V_1-Infer improves Pass@1 by up to 10% over pointwise verification and outperforms recent test-time scaling methods while being significantly more efficient. Furthermore, V_1-PairRL achieves 7--9% test-time scaling gains over standard RL and pointwise joint training, and improves base Pass@1 by up to 8.7% over standard RL in a code-generation setting.
Test-time scaling methods for complex reasoning tasks are enhanced through unified generation and verification frameworks that leverage pairwise self-verification, achieving improved performance in code generation and mathematical reasoning benchmarks.
test-time scaling complex reasoning tasks inference-time compute sampling aggregation verification scalar scoring pairwise self-verification V_1 framework uncertainty-guided algorithm tournament-based ranking dynamic allocation reinforcement learning joint training Pass@1 LiveCodeBench CodeContests SWE-Bench AIME HMMT