Insights from the ICLR Peer Review and Rebuttal Process
Peer review is a cornerstone of scientific publishing, including at premier machine learning conferences such as ICLR. As submission volumes increase, understanding the nature and dynamics of the review process is crucial for improving its efficiency, effectiveness, and the quality of published papers. We present a large-scale analysis of the ICLR 2024 and 2025 peer review processes, focusing on before- and after-rebuttal scores and reviewer-author interactions. We examine review scores, author-reviewer engagement, temporal patterns in review submissions, and co-reviewer influence effects. Combining quantitative analyses with LLM-based categorization of review texts and rebuttal discussions, we identify common strengths and weaknesses for each rating group, as well as trends in rebuttal strategies that are most strongly associated with score changes. Our findings show that initial scores and the ratings of co-reviewers are the strongest predictors of score changes during the rebuttal, pointing to a degree of reviewer influence. Rebuttals play a valuable role in improving outcomes for borderline papers, where thoughtful author responses can meaningfully shift reviewer perspectives. More broadly, our study offers evidence-based insights to improve the peer review process, guiding authors on effective rebuttal strategies and helping the community design fairer and more efficient review processes. Our code and score changes data are available at https://github.com/papercopilot/iclr-insights.
69242f90b5612535ed955a8e
Amir Hossein Kargaran
claimed_verified
2025-11-24T12:26:48.490Z
61bf84c8ca59d6d196a1b4e8
https://cdn-avatars.huggingface.co/v1/production/uploads/61bf84c8ca59d6d196a1b4e8/5CTo8gpBXKeKA5cbQLU7b.jpeg
Amir Hossein Kargaran
kargaranamir
user
kargaranamir
69242f90b5612535ed955a8f
Nafiseh Nikeghbal
claimed_verified
2026-04-16T20:23:09.189Z
62cb464b6193ba3ced7e5082
https://cdn-avatars.huggingface.co/v1/production/uploads/62cb464b6193ba3ced7e5082/CBovM4H7ESy3UuHk2etW9.jpeg
nikeghbal
nafisehNik
user
nafisehNik
69242f90b5612535ed955a90
Jing Yang
69242f90b5612535ed955a91
Nedjma Ousidhoum
Insights from the ICLR Peer Review and Rebuttal Process
Peer review is a cornerstone of scientific publishing, including at premier machine learning conferences such as ICLR. As submission volumes increase, understanding the nature and dynamics of the review process is crucial for improving its efficiency, effectiveness, and the quality of published papers. We present a large-scale analysis of the ICLR 2024 and 2025 peer review processes, focusing on before- and after-rebuttal scores and reviewer-author interactions. We examine review scores, author-reviewer engagement, temporal patterns in review submissions, and co-reviewer influence effects. Combining quantitative analyses with LLM-based categorization of review texts and rebuttal discussions, we identify common strengths and weaknesses for each rating group, as well as trends in rebuttal strategies that are most strongly associated with score changes. Our findings show that initial scores and the ratings of co-reviewers are the strongest predictors of score changes during the rebuttal, pointing to a degree of reviewer influence. Rebuttals play a valuable role in improving outcomes for borderline papers, where thoughtful author responses can meaningfully shift reviewer perspectives. More broadly, our study offers evidence-based insights to improve the peer review process, guiding authors on effective rebuttal strategies and helping the community design fairer and more efficient review processes. Our code and score changes data are available at https://github.com/papercopilot/iclr-insights.
The study analyzes the ICLR 2024 and 2025 peer review processes, focusing on score dynamics and reviewer interactions, using LLM-based text categorization to identify trends and factors influencing score changes.
peer review ICLR review scores author-reviewer engagement temporal patterns co-reviewer influence LLM-based categorization rebuttal strategies score changes reviewer influence