RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation
Large language models (LLMs) are increasingly used across the scientific workflow, including to draft peer-review reports. However, many AI-generated reviews are superficial and insufficiently actionable, leaving authors without concrete, implementable guidance and motivating the gap this work addresses. We propose RbtAct, which targets actionable review feedback generation and places existing peer review rebuttal at the center of learning. Rebuttals show which reviewer comments led to concrete revisions or specific plans, and which were only defended. Building on this insight, we leverage rebuttal as implicit supervision to directly optimize a feedback generator for actionability. To support this objective, we propose a new task called perspective-conditioned segment-level review feedback generation, in which the model is required to produce a single focused comment based on the complete paper and a specified perspective such as experiments and writing. We also build a large dataset named RMR-75K that maps review segments to the rebuttal segments that address them, with perspective labels and impact categories that order author uptake. We then train the Llama-3.1-8B-Instruct model with supervised fine-tuning on review segments followed by preference optimization using rebuttal derived pairs. Experiments with human experts and LLM-as-a-judge show consistent gains in actionability and specificity over strong baselines while maintaining grounding and relevance.
69b1718fef3e680564226813
Sihong Wu
admin_assigned
2026-03-12T10:43:33.927Z
67911b57692e0b97d9936ee3
/avatars/270cb16773b0ba2a1fde8e7e18979971.svg
Sihong Wu
shwu22
user
shwu22
69b1718fef3e680564226814
Yiling Ma
admin_assigned
2026-03-12T10:43:46.358Z
69227307168e7fa5935fc7a6
/avatars/20f959a3314ce098fcd224ae999e2a8f.svg
YilingMa
YilingMa
user
YilingMa
69b1718fef3e680564226815
Yilun Zhao
admin_assigned
2026-03-12T10:43:52.814Z
62f662bcc58915315c4eccea
https://cdn-avatars.huggingface.co/v1/production/uploads/62f662bcc58915315c4eccea/zOAQLONfMP88zr70sxHK-.jpeg
Yilun Zhao
yilunzhao
user
yilunzhao
69b1718fef3e680564226816
Tiansheng Hu
claimed_verified
2026-03-12T09:07:43.017Z
67492b9e347c3876f22b3684
/avatars/0d80d23f7b10ce8bac689f6e8317a014.svg
Tiansheng Hu
HughieHu
user
HughieHu
69b1718fef3e680564226817
Owen Jiang
69b1718fef3e680564226818
Manasi Patwardhan
admin_assigned
2026-03-12T10:43:58.419Z
60c0f00a04b4c02707612591
/avatars/e35bb8d22261855bb10f746277a8c11f.svg
Manasi Samarth Patwardhan
ManasiPatwardhan
user
ManasiPatwardhan
69b1718fef3e680564226819
Arman Cohan
admin_assigned
2026-03-12T10:44:05.314Z
5f5ba21188f57f65f951f255
https://cdn-avatars.huggingface.co/v1/production/uploads/1599840760465-noauth.png
Arman Cohan
armanc
user
armanc
RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation
Large language models (LLMs) are increasingly used across the scientific workflow, including to draft peer-review reports. However, many AI-generated reviews are superficial and insufficiently actionable, leaving authors without concrete, implementable guidance and motivating the gap this work addresses. We propose RbtAct, which targets actionable review feedback generation and places existing peer review rebuttal at the center of learning. Rebuttals show which reviewer comments led to concrete revisions or specific plans, and which were only defended. Building on this insight, we leverage rebuttal as implicit supervision to directly optimize a feedback generator for actionability. To support this objective, we propose a new task called perspective-conditioned segment-level review feedback generation, in which the model is required to produce a single focused comment based on the complete paper and a specified perspective such as experiments and writing. We also build a large dataset named RMR-75K that maps review segments to the rebuttal segments that address them, with perspective labels and impact categories that order author uptake. We then train the Llama-3.1-8B-Instruct model with supervised fine-tuning on review segments followed by preference optimization using rebuttal derived pairs. Experiments with human experts and LLM-as-a-judge show consistent gains in actionability and specificity over strong baselines while maintaining grounding and relevance.
Researchers developed RbtAct, a method that uses rebuttal responses to improve the actionability of AI-generated peer-review feedback by training a language model to produce specific, implementable comments based on paper content and defined perspectives.
large language models peer-review reports rebuttal supervised fine-tuning preference optimization perspective-conditioned segment-level review feedback generation RMR-75K dataset Llama-3.1-8B-Instruct