Interactive Benchmarks
Standard benchmarks have become increasingly unreliable due to saturation, subjectivity, and poor generalization. We argue that evaluating model's ability to acquire information actively is important to assess model's intelligence. We propose Interactive Benchmarks, a unified evaluation paradigm that assesses model's reasoning ability in an interactive process under budget constraints. We instantiate this framework across two settings: Interactive Proofs, where models interact with a judge to deduce objective truths or answers in logic and mathematics; and Interactive Games, where models reason strategically to maximize long-horizon utilities. Our results show that interactive benchmarks provide a robust and faithful assessment of model intelligence, revealing that there is still substantial room to improve in interactive scenarios. Project page: https://github.com/interactivebench/interactivebench
69aae659a0a83914fb8e126e
Baoqing Yue
69aae659a0a83914fb8e126f
Zihan Zhu
69aae659a0a83914fb8e1270
Yifan Zhang
claimed_verified
2026-03-09T09:58:18.647Z
647bf082aba7062fe5c51ca9
https://cdn-avatars.huggingface.co/v1/production/uploads/647bf082aba7062fe5c51ca9/VvKAhQC_LxBcBuy3XROSX.jpeg
Yifan Zhang
yifAI
user
yifAI
69aae659a0a83914fb8e1271
Jichen Feng
69aae659a0a83914fb8e1272
Hufei Yang
69aae659a0a83914fb8e1273
Mengdi Wang
Interactive Benchmarks
Standard benchmarks have become increasingly unreliable due to saturation, subjectivity, and poor generalization. We argue that evaluating model's ability to acquire information actively is important to assess model's intelligence. We propose Interactive Benchmarks, a unified evaluation paradigm that assesses model's reasoning ability in an interactive process under budget constraints. We instantiate this framework across two settings: Interactive Proofs, where models interact with a judge to deduce objective truths or answers in logic and mathematics; and Interactive Games, where models reason strategically to maximize long-horizon utilities. Our results show that interactive benchmarks provide a robust and faithful assessment of model intelligence, revealing that there is still substantial room to improve in interactive scenarios. Project page: https://github.com/interactivebench/interactivebench
Interactive benchmarks offer a unified framework for evaluating model intelligence through active information acquisition under constraint conditions, demonstrating superior assessment of reasoning capabilities compared to traditional benchmarks.
interactive benchmarks model intelligence active information acquisition reasoning ability budget constraints interactive proofs interactive games model evaluation