PRISM: Pushing the Frontier of Deep Think via Process Reward Model-Guided Inference
DEEPTHINK methods improve reasoning by generating, refining, and aggregating populations of candidate solutions, which enables strong performance on complex mathematical and scientific tasks. However, existing frameworks often lack reliable correctness signals during inference, which creates a population-enhancement bottleneck where deeper deliberation amplifies errors, suppresses correct minority solutions, and yields weak returns to additional compute. In this paper, we introduce a functional decomposition of DEEPTHINK systems and propose PRISM, a Process Reward Model (PRM)-guided inference algorithm that uses step-level verification to guide both population refinement and solution aggregation. During refinement, PRISM treats candidate solutions as particles in a PRM-defined energy landscape and reshapes the population through score-guided resampling and stochastic refinement, which concentrates probability mass on higher-quality reasoning while preserving diversity. Across mathematics and science benchmarks, PRISM is competitive with or outperforms existing DEEPTHINK methods, reaching 90.0%, 75.4%, and 71.4% with gpt-oss-20b on AIME25, HMMT25, and GPQA Diamond, respectively, while matching or exceeding gpt-oss-120b. Additionally, our analysis shows that PRISM produces consistent net-directional correction during refinement, remains reliable when the initial population contains few correct candidates, and often lies on the compute-accuracy Pareto frontier.
69a79fb52833ff1122ff6427
Rituraj Sharma
69a79fb52833ff1122ff6428
Weiyuan Chen
claimed_verified
2026-03-05T09:59:23.634Z
6652e84bff6ccc0ef5ac7055
/avatars/70a8b6a0bae5f9e954fa15f56ab2ddc3.svg
Weiyuan Chen
WeiyuanChen01
user
WeiyuanChen01
69a79fb52833ff1122ff6429
Noah Provenzano
69a79fb52833ff1122ff642a
Tu Vu
claimed_verified
2026-03-04T09:22:49.234Z
682f7c792ec6a13f2a5eda14
/avatars/c2c198c7ab0182bb279cc82f87498797.svg
Tu Vu
tuvllms
user
tuvllms
PRISM: Pushing the Frontier of Deep Think via Process Reward Model-Guided Inference
DEEPTHINK methods improve reasoning by generating, refining, and aggregating populations of candidate solutions, which enables strong performance on complex mathematical and scientific tasks. However, existing frameworks often lack reliable correctness signals during inference, which creates a population-enhancement bottleneck where deeper deliberation amplifies errors, suppresses correct minority solutions, and yields weak returns to additional compute. In this paper, we introduce a functional decomposition of DEEPTHINK systems and propose PRISM, a Process Reward Model (PRM)-guided inference algorithm that uses step-level verification to guide both population refinement and solution aggregation. During refinement, PRISM treats candidate solutions as particles in a PRM-defined energy landscape and reshapes the population through score-guided resampling and stochastic refinement, which concentrates probability mass on higher-quality reasoning while preserving diversity. Across mathematics and science benchmarks, PRISM is competitive with or outperforms existing DEEPTHINK methods, reaching 90.0%, 75.4%, and 71.4% with gpt-oss-20b on AIME25, HMMT25, and GPQA Diamond, respectively, while matching or exceeding gpt-oss-120b. Additionally, our analysis shows that PRISM produces consistent net-directional correction during refinement, remains reliable when the initial population contains few correct candidates, and often lies on the compute-accuracy Pareto frontier.
PRISM is a Process Reward Model-guided inference algorithm that enhances DEEPTHINK systems by using step-level verification to improve population refinement and solution aggregation, achieving strong performance on mathematical and scientific benchmarks.
DEEPTHINK candidate solutions population enhancement reasoning Process Reward Model step-level verification score-guided resampling stochastic refinement energy landscape compute-accuracy Pareto frontier