Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
Recent progress in large language models (LLMs) has led to impressive performance on a range of tasks, yet advanced instruction following (IF)-especially for complex, multi-turn, and system-prompted instructions-remains a significant challenge. Rigorous evaluation and effective training for such capabilities are hindered by the lack of high-quality, human-annotated benchmarks and reliable, interpretable reward signals. In this work, we introduce AdvancedIF (we will release this benchmark soon), a comprehensive benchmark featuring over 1,600 prompts and expert-curated rubrics that assess LLMs ability to follow complex, multi-turn, and system-level instructions. We further propose RIFL (Rubric-based Instruction-Following Learning), a novel post-training pipeline that leverages rubric generation, a finetuned rubric verifier, and reward shaping to enable effective reinforcement learning for instruction following. Extensive experiments demonstrate that RIFL substantially improves the instruction-following abilities of LLMs, achieving a 6.7% absolute gain on AdvancedIF and strong results on public benchmarks. Our ablation studies confirm the effectiveness of each component in RIFL. This work establishes rubrics as a powerful tool for both training and evaluating advanced IF in LLMs, paving the way for more capable and reliable AI systems.
691739c1a6cea0a2e76c970d
Yun He
691739c1a6cea0a2e76c970e
Wenzhe Li
691739c1a6cea0a2e76c970f
Hejia Zhang
691739c1a6cea0a2e76c9710
Songlin Li
claimed_verified
2026-02-20T08:38:35.761Z
678d60dae7b3986f59869666
/avatars/c78e30338af124001043191d1f4f864c.svg
Songlin Li
vincent-sk-li
user
vincent-sk-li
691739c1a6cea0a2e76c9711
Karishma Mandyam
691739c1a6cea0a2e76c9712
Sopan Khosla
691739c1a6cea0a2e76c9713
Yuanhao Xiong
691739c1a6cea0a2e76c9714
Nanshu Wang
691739c1a6cea0a2e76c9715
Selina Peng
691739c1a6cea0a2e76c9716
Beibin Li
691739c1a6cea0a2e76c9717
Shengjie Bi
691739c1a6cea0a2e76c9718
Shishir G. Patil
691739c1a6cea0a2e76c9719
Qi Qi
691739c1a6cea0a2e76c971a
Shengyu Feng
691739c1a6cea0a2e76c971b
Julian Katz-Samuels
691739c1a6cea0a2e76c971c
Richard Yuanzhe Pang
691739c1a6cea0a2e76c971d
Sujan Gonugondla
691739c1a6cea0a2e76c971e
Hunter Lang
691739c1a6cea0a2e76c971f
Yue Yu
691739c1a6cea0a2e76c9720
Yundi Qian
691739c1a6cea0a2e76c9721
Maryam Fazel-Zarandi
691739c1a6cea0a2e76c9722
Licheng Yu
691739c1a6cea0a2e76c9723
Amine Benhalloum
691739c1a6cea0a2e76c9724
Hany Awadalla
691739c1a6cea0a2e76c9725
Manaal Faruqui
Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
Recent progress in large language models (LLMs) has led to impressive performance on a range of tasks, yet advanced instruction following (IF)-especially for complex, multi-turn, and system-prompted instructions-remains a significant challenge. Rigorous evaluation and effective training for such capabilities are hindered by the lack of high-quality, human-annotated benchmarks and reliable, interpretable reward signals. In this work, we introduce AdvancedIF (we will release this benchmark soon), a comprehensive benchmark featuring over 1,600 prompts and expert-curated rubrics that assess LLMs ability to follow complex, multi-turn, and system-level instructions. We further propose RIFL (Rubric-based Instruction-Following Learning), a novel post-training pipeline that leverages rubric generation, a finetuned rubric verifier, and reward shaping to enable effective reinforcement learning for instruction following. Extensive experiments demonstrate that RIFL substantially improves the instruction-following abilities of LLMs, achieving a 6.7% absolute gain on AdvancedIF and strong results on public benchmarks. Our ablation studies confirm the effectiveness of each component in RIFL. This work establishes rubrics as a powerful tool for both training and evaluating advanced IF in LLMs, paving the way for more capable and reliable AI systems.
AdvancedIF benchmark and RIFL pipeline improve instruction-following capabilities in large language models by using expert-curated rubrics and reinforcement learning techniques.
large language models (LLMs) instruction following (IF) multi-turn instructions system-prompted instructions human-annotated benchmarks rubrics reinforcement learning rubric generation rubric verifier reward shaping