Reasoning-Aware GRPO using Process Mining
Reinforcement learning (RL)-based post-training has been crucial for enabling
multi-step reasoning in large reasoning models (LRMs), yet current reward
schemes are typically outcome-centric. We propose PM4GRPO, a reasoning-aware
Group Relative Policy Optimization (GRPO) that augments standard answer/format
rewards with signals over the reasoning procedure. To this end, process mining
techniques are utilized to compute a scalar conformance reward that measures
how closely a policy model's reasoning aligns with the pretrained teacher
model. The empirical results on five benchmarks demonstrate that PM4GRPO
significantly outperforms existing methodologies for GRPO-based post-training.
These results highlight that leveraging process mining for reasoning-aware GRPO
effectively enhances the reasoning capabilities of policy models.
6902cbef72739622ee92a7bd
Taekhyun Park
claimed_verified
2025-10-30T14:28:15.794Z
64ba75761d0a5a5760874197
https://cdn-avatars.huggingface.co/v1/production/uploads/64ba75761d0a5a5760874197/5V3jxX-vim-wMaU2Zjz5h.jpeg
TaekHyunPark
Thrillcrazyer
user
Thrillcrazyer
6902cbef72739622ee92a7be
Yongjae Lee
claimed_verified
2025-10-30T14:28:12.519Z
67add23f4233b0da556921a7
/avatars/f2b8b1c42e463b67c01e23ee9bf177c0.svg
Lee
yongzzzai
user
yongzzzai
6902cbef72739622ee92a7bf
Hyerim Bae
Reasoning-Aware GRPO using Process Mining
Reinforcement learning (RL)-based post-training has been crucial for enabling
multi-step reasoning in large reasoning models (LRMs), yet current reward
schemes are typically outcome-centric. We propose PM4GRPO, a reasoning-aware
Group Relative Policy Optimization (GRPO) that augments standard answer/format
rewards with signals over the reasoning procedure. To this end, process mining
techniques are utilized to compute a scalar conformance reward that measures
how closely a policy model's reasoning aligns with the pretrained teacher
model. The empirical results on five benchmarks demonstrate that PM4GRPO
significantly outperforms existing methodologies for GRPO-based post-training.
These results highlight that leveraging process mining for reasoning-aware GRPO
effectively enhances the reasoning capabilities of policy models.
PM4GRPO, a reasoning-aware Group Relative Policy Optimization, enhances policy models by incorporating process mining to align reasoning with a teacher model, outperforming existing methods.
reinforcement learning multi-step reasoning large reasoning models reasoning-aware Group Relative Policy Optimization GRPO process mining scalar conformance reward pretrained teacher model