SWE-RM: Execution-free Feedback For Software Engineering Agents
Execution-based feedback like unit testing is widely used in the development of coding agents through test-time scaling (TTS) and reinforcement learning (RL). This paradigm requires scalable and reliable collection of unit test cases to provide accurate feedback, and the resulting feedback is often sparse and cannot effectively distinguish between trajectories that are both successful or both unsuccessful. In contrast, execution-free feedback from reward models can provide more fine-grained signals without depending on unit test cases. Despite this potential, execution-free feedback for realistic software engineering (SWE) agents remains underexplored. Aiming to develop versatile reward models that are effective across TTS and RL, however, we observe that two verifiers with nearly identical TTS performance can nevertheless yield very different results in RL. Intuitively, TTS primarily reflects the model's ability to select the best trajectory, but this ability does not necessarily generalize to RL. To address this limitation, we identify two additional aspects that are crucial for RL training: classification accuracy and calibration. We then conduct comprehensive controlled experiments to investigate how to train a robust reward model that performs well across these metrics. In particular, we analyze the impact of various factors such as training data scale, policy mixtures, and data source composition. Guided by these investigations, we introduce SWE-RM, an accurate and robust reward model adopting a mixture-of-experts architecture with 30B total parameters and 3B activated during inference. SWE-RM substantially improves SWE agents on both TTS and RL performance. For example, it increases the accuracy of Qwen3-Coder-Flash from 51.6% to 62.0%, and Qwen3-Coder-Max from 67.0% to 74.6% on SWE-Bench Verified using TTS, achieving new state-of-the-art performance among open-source models.
6951e7bc746a34b55dd5489c
KaShun Shum
6951e7bc746a34b55dd5489d
Binyuan Hui
6951e7bc746a34b55dd5489e
Jiawei Chen
6951e7bc746a34b55dd5489f
Lei Zhang
claimed_verified
2026-03-23T11:02:28.881Z
60fcff6053d9635123d88452
https://cdn-avatars.huggingface.co/v1/production/uploads/60fcff6053d9635123d88452/QnVvuQyODVHyo7aR89kKA.jpeg
Lei Zhang
Hambaobao
user
Hambaobao
6951e7bc746a34b55dd548a0
X. W.
6951e7bc746a34b55dd548a1
Jiaxi Yang
claimed_verified
2026-02-03T10:09:37.194Z
646df403ad20c6fa4f30b7ec
https://cdn-avatars.huggingface.co/v1/production/uploads/646df403ad20c6fa4f30b7ec/Q64-XMghOcBoo3itZDGYA.jpeg
Jiaxi Yang
jx-yang
user
jx-yang
6951e7bc746a34b55dd548a2
Yuzhen Huang
claimed_verified
2025-12-31T20:57:33.380Z
6462def82a83863b97c0611e
/avatars/c03e9cc7d75b0266fcc56ecb6ee62148.svg
Yuzhen Huang
yuzhen17
user
yuzhen17
6951e7bc746a34b55dd548a3
Junyang Lin
6951e7bc746a34b55dd548a4
Junxian He
SWE-RM: Execution-free Feedback For Software Engineering Agents
Execution-based feedback like unit testing is widely used in the development of coding agents through test-time scaling (TTS) and reinforcement learning (RL). This paradigm requires scalable and reliable collection of unit test cases to provide accurate feedback, and the resulting feedback is often sparse and cannot effectively distinguish between trajectories that are both successful or both unsuccessful. In contrast, execution-free feedback from reward models can provide more fine-grained signals without depending on unit test cases. Despite this potential, execution-free feedback for realistic software engineering (SWE) agents remains underexplored. Aiming to develop versatile reward models that are effective across TTS and RL, however, we observe that two verifiers with nearly identical TTS performance can nevertheless yield very different results in RL. Intuitively, TTS primarily reflects the model's ability to select the best trajectory, but this ability does not necessarily generalize to RL. To address this limitation, we identify two additional aspects that are crucial for RL training: classification accuracy and calibration. We then conduct comprehensive controlled experiments to investigate how to train a robust reward model that performs well across these metrics. In particular, we analyze the impact of various factors such as training data scale, policy mixtures, and data source composition. Guided by these investigations, we introduce SWE-RM, an accurate and robust reward model adopting a mixture-of-experts architecture with 30B total parameters and 3B activated during inference. SWE-RM substantially improves SWE agents on both TTS and RL performance. For example, it increases the accuracy of Qwen3-Coder-Flash from 51.6% to 62.0%, and Qwen3-Coder-Max from 67.0% to 74.6% on SWE-Bench Verified using TTS, achieving new state-of-the-art performance among open-source models.