ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development
The evolution of Large Language Models (LLMs) into autonomous agents has expanded the scope of AI coding from localized code generation to complex, repository-level, and execution-driven problem solving. However, current benchmarks predominantly evaluate code logic in static contexts, neglecting the dynamic, full-process requirements of real-world engineering, particularly in backend development which demands rigorous environment configuration and service deployment. To address this gap, we introduce ABC-Bench, a benchmark explicitly designed to evaluate agentic backend coding within a realistic, executable workflow. Using a scalable automated pipeline, we curated 224 practical tasks spanning 8 languages and 19 frameworks from open-source repositories. Distinct from previous evaluations, ABC-Bench require the agents to manage the entire development lifecycle from repository exploration to instantiating containerized services and pass the external end-to-end API tests. Our extensive evaluation reveals that even state-of-the-art models struggle to deliver reliable performance on these holistic tasks, highlighting a substantial disparity between current model capabilities and the demands of practical backend engineering. Our code is available at https://github.com/OpenMOSS/ABC-Bench.
696da04e3f1837bfb89709c2
Jie Yang
claimed_verified
2026-01-20T09:41:07.348Z
66206a2136201a18e5329631
https://cdn-avatars.huggingface.co/v1/production/uploads/66206a2136201a18e5329631/9aJiGOKshh1tZ171zDw_D.png
yangjie
red-fox-yj
user
red-fox-yj
696da04e3f1837bfb89709c3
Honglin Guo
claimed_verified
2026-01-20T09:41:05.334Z
638ef0b0c67af472d31674a6
https://cdn-avatars.huggingface.co/v1/production/uploads/638ef0b0c67af472d31674a6/zXQjC3DdY3jpVkATkpms6.png
Honglin Guo
KYLN24
user
KYLN24
696da04e3f1837bfb89709c4
Li Ji
696da04e3f1837bfb89709c5
Jiazheng Zhou
admin_assigned
2026-01-20T09:42:49.533Z
683c6a19a4b3e38a3e23d50a
/avatars/ae9f212acaa9d1a65b4a5d86c5f7a355.svg
Jiazheng Zhou
HaZ-K
user
HaZ-K
696da04e3f1837bfb89709c6
Rui Zheng
696da04e3f1837bfb89709c7
Zhikai Lei
696da04e3f1837bfb89709c8
Shuo Zhang
claimed_verified
2026-01-20T14:48:19.478Z
6334f2f1259c518276efa730
https://cdn-avatars.huggingface.co/v1/production/uploads/6334f2f1259c518276efa730/z_SH_OBkDyj4RCN9mqsKS.jpeg
Shuo Zhang
Meteonis
user
Meteonis
696da04e3f1837bfb89709c9
Zhiheng Xi
admin_assigned
2026-01-20T09:43:14.561Z
653a6e5cae155b92bae77b74
https://cdn-avatars.huggingface.co/v1/production/uploads/653a6e5cae155b92bae77b74/TA5FWKAUsB249ux4MzD_R.jpeg
Zhiheng Xi
WooooDyy
user
WooooDyy
696da04e3f1837bfb89709ca
Shichun Liu
admin_assigned
2026-01-20T09:43:20.286Z
65435cad429b80b14922ab8d
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/N8oWq4ZZn3dRxmXi18FrA.jpeg
Shichun Liu
Liusc2020
user
Liusc2020
696da04e3f1837bfb89709cb
Yuxin Wang
696da04e3f1837bfb89709cc
Bo Wang
claimed_verified
2026-04-15T12:10:19.475Z
663c23b4a4152b77b67b7c1c
/avatars/7af03530ce2b48bc77ce84296878978f.svg
Bo Wang
Musicode
user
Musicode
696da04e3f1837bfb89709cd
Yining Zheng
admin_assigned
2026-01-20T09:43:29.765Z
64b7495a75b23e68c538f4c0
/avatars/ce06f3b89f9e09dcbe748b208eec1e9d.svg
Yining Zheng
WillQvQ
user
WillQvQ
696da04e3f1837bfb89709ce
Tao Gui
696da04e3f1837bfb89709cf
Xipeng Qiu
admin_assigned
2026-01-20T09:43:35.123Z
61457b8deff2c9fdb4de4988
https://cdn-avatars.huggingface.co/v1/production/uploads/1632381702899-61457b8deff2c9fdb4de4988.jpeg
Xipeng Qiu
xpqiu
user
xpqiu
ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development
The evolution of Large Language Models (LLMs) into autonomous agents has expanded the scope of AI coding from localized code generation to complex, repository-level, and execution-driven problem solving. However, current benchmarks predominantly evaluate code logic in static contexts, neglecting the dynamic, full-process requirements of real-world engineering, particularly in backend development which demands rigorous environment configuration and service deployment. To address this gap, we introduce ABC-Bench, a benchmark explicitly designed to evaluate agentic backend coding within a realistic, executable workflow. Using a scalable automated pipeline, we curated 224 practical tasks spanning 8 languages and 19 frameworks from open-source repositories. Distinct from previous evaluations, ABC-Bench require the agents to manage the entire development lifecycle from repository exploration to instantiating containerized services and pass the external end-to-end API tests. Our extensive evaluation reveals that even state-of-the-art models struggle to deliver reliable performance on these holistic tasks, highlighting a substantial disparity between current model capabilities and the demands of practical backend engineering. Our code is available at https://github.com/OpenMOSS/ABC-Bench.
ABC-Bench evaluates LLM agents on realistic backend coding tasks requiring full development lifecycle management from repository exploration to containerized service deployment and API testing.
Large Language Models agentic backend coding executable workflow development lifecycle containerized services end-to-end API tests