FrontierCS: Evolving Challenges for Evolving Intelligence
We introduce FrontierCS, a benchmark of 156 open-ended problems across diverse areas of computer science, designed and reviewed by experts, including CS PhDs and top-tier competitive programming participants and problem setters. Unlike existing benchmarks that focus on tasks with known optimal solutions, FrontierCS targets problems where the optimal solution is unknown, but the quality of a solution can be objectively evaluated. Models solve these tasks by implementing executable programs rather than outputting a direct answer. FrontierCS includes algorithmic problems, which are often NP-hard variants of competitive programming problems with objective partial scoring, and research problems with the same property. For each problem we provide an expert reference solution and an automatic evaluator. Combining open-ended design, measurable progress, and expert curation, FrontierCS provides a benchmark at the frontier of computer-science difficulty. Empirically, we find that frontier reasoning models still lag far behind human experts on both the algorithmic and research tracks, that increasing reasoning budgets alone does not close this gap, and that models often over-optimize for generating merely workable code instead of discovering high-quality algorithms and system designs.
69446a6bfbf17e708e185e64
Qiuyang Mang
claimed_verified
2026-03-13T08:15:43.408Z
69446b15835f00df604cbc7a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/203QH9qkSLhigPnPA2SZW.webp
Qiuyang Mang
qmang
user
qmang
69446a6bfbf17e708e185e65
Wenhao Chai
69446a6bfbf17e708e185e66
Zhifei Li
69446a6bfbf17e708e185e67
Huanzhi Mao
claimed_verified
2025-12-19T08:58:30.924Z
65bf1f053b879593a57d4e6a
https://cdn-avatars.huggingface.co/v1/production/uploads/65bf1f053b879593a57d4e6a/WiHAU6jvRtW0En-hRK3MI.jpeg
HuanzhiMao
HuanzhiMao
user
HuanzhiMao
69446a6bfbf17e708e185e68
Shang Zhou
69446a6bfbf17e708e185e69
Alexander Du
69446a6bfbf17e708e185e6a
Hanchen Li
69446a6bfbf17e708e185e6b
Shu Liu
69446a6bfbf17e708e185e6c
Edwin Chen
69446a6bfbf17e708e185e6d
Yichuan Wang
69446a6bfbf17e708e185e6e
Xieting Chu
69446a6bfbf17e708e185e6f
Zerui Cheng
69446a6bfbf17e708e185e70
Yuan Xu
69446a6bfbf17e708e185e71
Tian Xia
69446a6bfbf17e708e185e72
Zirui Wang
claimed_verified
2026-01-26T08:34:07.526Z
641a38fdfb5ffff5ac78ceb0
https://cdn-avatars.huggingface.co/v1/production/uploads/641a38fdfb5ffff5ac78ceb0/53nGstZ9Ya5WfeGdXAXCr.png
Zirui Wang
zwcolin
user
zwcolin
69446a6bfbf17e708e185e73
Tianneng Shi
69446a6bfbf17e708e185e74
Jianzhu Yao
69446a6bfbf17e708e185e75
Yilong Zhao
69446a6bfbf17e708e185e76
Qizheng Zhang
claimed_verified
2026-04-13T07:39:09.085Z
6663e5b10c54dffcd2a921ca
/avatars/2a4589fef05306ccde06728c752e5601.svg
Qizheng Zhang
qizhengz
user
qizhengz
69446a6bfbf17e708e185e77
Charlie Ruan
69446a6bfbf17e708e185e78
Zeyu Shen
69446a6bfbf17e708e185e79
Kaiyuan Liu
69446a6bfbf17e708e185e7a
Runyuan He
69446a6bfbf17e708e185e7b
Dong Xing
69446a6bfbf17e708e185e7c
Zerui Li
69446a6bfbf17e708e185e7d
Zirong Zeng
69446a6bfbf17e708e185e7e
Yige Jiang
69446a6bfbf17e708e185e7f
Lufeng Cheng
69446a6bfbf17e708e185e80
Ziyi Zhao
69446a6bfbf17e708e185e81
Youran Sun
69446a6bfbf17e708e185e82
Wesley Zheng
69446a6bfbf17e708e185e83
Meiyuwang Zhang
69446a6bfbf17e708e185e84
Ruyi Ji
69446a6bfbf17e708e185e85
Xuechang Tu
69446a6bfbf17e708e185e86
Zihan Zheng
69446a6bfbf17e708e185e87
Zexing Chen
69446a6bfbf17e708e185e88
Kangyang Zhou
69446a6bfbf17e708e185e89
Zhaozi Wang
69446a6bfbf17e708e185e8a
Jingbang Chen
69446a6bfbf17e708e185e8b
Aleksandra Korolova
69446a6bfbf17e708e185e8c
Peter Henderson
69446a6bfbf17e708e185e8d
Pramod Viswanath
69446a6bfbf17e708e185e8e
Vijay Ganesh
69446a6bfbf17e708e185e8f
Saining Xie
69446a6bfbf17e708e185e90
Zhuang Liu
69446a6bfbf17e708e185e91
Dawn Song
69446a6bfbf17e708e185e92
Sewon Min
69446a6bfbf17e708e185e93
Ion Stoica
69446a6bfbf17e708e185e94
Joseph E. Gonzalez
69446a6bfbf17e708e185e95
Jingbo Shang
69446a6bfbf17e708e185e96
Alvin Cheung
FrontierCS: Evolving Challenges for Evolving Intelligence
We introduce FrontierCS, a benchmark of 156 open-ended problems across diverse areas of computer science, designed and reviewed by experts, including CS PhDs and top-tier competitive programming participants and problem setters. Unlike existing benchmarks that focus on tasks with known optimal solutions, FrontierCS targets problems where the optimal solution is unknown, but the quality of a solution can be objectively evaluated. Models solve these tasks by implementing executable programs rather than outputting a direct answer. FrontierCS includes algorithmic problems, which are often NP-hard variants of competitive programming problems with objective partial scoring, and research problems with the same property. For each problem we provide an expert reference solution and an automatic evaluator. Combining open-ended design, measurable progress, and expert curation, FrontierCS provides a benchmark at the frontier of computer-science difficulty. Empirically, we find that frontier reasoning models still lag far behind human experts on both the algorithmic and research tracks, that increasing reasoning budgets alone does not close this gap, and that models often over-optimize for generating merely workable code instead of discovering high-quality algorithms and system designs.
FrontierCS is a benchmark for evaluating models on open-ended computer science problems with unknown optimal solutions, where tasks involve implementing executable programs.
algorithmic problems NP-hard competitive programming objective partial scoring research problems expert reference solution automatic evaluator frontier reasoning models