Accurate Failure Prediction in Agents Does Not Imply Effective Failure Prevention
Proactive interventions by LLM critic models are often assumed to improve reliability, yet their effects at deployment time are poorly understood. We show that a binary LLM critic with strong offline accuracy (AUROC 0.94) can nevertheless cause severe performance degradation, inducing a 26 percentage point (pp) collapse on one model while affecting another by near zero pp. This variability demonstrates that LLM critic accuracy alone is insufficient to determine whether intervention is safe.
  We identify a disruption-recovery tradeoff: interventions may recover failing trajectories but also disrupt trajectories that would have succeeded. Based on this insight, we propose a pre-deployment test that uses a small pilot of 50 tasks to estimate whether intervention is likely to help or harm, without requiring full deployment. Across benchmarks, the test correctly anticipates outcomes: intervention degrades performance on high-success tasks (0 to -26 pp), while yielding a modest improvement on the high-failure ALFWorld benchmark (+2.8 pp, p=0.014). The primary value of our framework is therefore identifying when not to intervene, preventing severe regressions before deployment.
6985a0ce4ad556f294b7ed16
Rakshith Vasudev
6985a0ce4ad556f294b7ed17
Melisa Russak
claimed_verified
2026-02-06T18:50:23.513Z
60e61b3969bd0df25c9375da
https://cdn-avatars.huggingface.co/v1/production/uploads/1625692968400-noauth.jpeg
Melisa Russak
melisa
user
melisa
6985a0ce4ad556f294b7ed18
Dan Bikel
6985a0ce4ad556f294b7ed19
Waseem Alshikh
claimed_verified
2026-02-09T08:35:07.222Z
60cd486d723acf5eb46fe8d3
https://cdn-avatars.huggingface.co/v1/production/uploads/60cd486d723acf5eb46fe8d3/Z1bD1kjvZ0QAOjZna41Xr.jpeg
Waseem AlShikh
wassemgtk
user
wassemgtk
Accurate Failure Prediction in Agents Does Not Imply Effective Failure Prevention
Proactive interventions by LLM critic models are often assumed to improve reliability, yet their effects at deployment time are poorly understood. We show that a binary LLM critic with strong offline accuracy (AUROC 0.94) can nevertheless cause severe performance degradation, inducing a 26 percentage point (pp) collapse on one model while affecting another by near zero pp. This variability demonstrates that LLM critic accuracy alone is insufficient to determine whether intervention is safe.
  We identify a disruption-recovery tradeoff: interventions may recover failing trajectories but also disrupt trajectories that would have succeeded. Based on this insight, we propose a pre-deployment test that uses a small pilot of 50 tasks to estimate whether intervention is likely to help or harm, without requiring full deployment. Across benchmarks, the test correctly anticipates outcomes: intervention degrades performance on high-success tasks (0 to -26 pp), while yielding a modest improvement on the high-failure ALFWorld benchmark (+2.8 pp, p=0.014). The primary value of our framework is therefore identifying when not to intervene, preventing severe regressions before deployment.
LLM critic models with high offline accuracy can cause variable performance impacts at deployment, necessitating pre-deployment testing to determine intervention safety and effectiveness.
LLM critic models AUROC proactive interventions deployment time disruption-recovery tradeoff pre-deployment test pilot testing ALFWorld benchmark