ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning
The rapid advancement of Large Language Models (LLMs) has led to performance saturation on many established benchmarks, questioning their ability to distinguish frontier models. Concurrently, existing high-difficulty benchmarks often suffer from narrow disciplinary focus, oversimplified answer formats, and vulnerability to data contamination, creating a fidelity gap with real-world scientific inquiry. To address these challenges, we introduce ATLAS (AGI-Oriented Testbed for Logical Application in Science), a large-scale, high-difficulty, and cross-disciplinary evaluation suite composed of approximately 800 original problems. Developed by domain experts (PhD-level and above), ATLAS spans seven core scientific fields: mathematics, physics, chemistry, biology, computer science, earth science, and materials science. Its key features include: (1) High Originality and Contamination Resistance, with all questions newly created or substantially adapted to prevent test data leakage; (2) Cross-Disciplinary Focus, designed to assess models' ability to integrate knowledge and reason across scientific domains; (3) High-Fidelity Answers, prioritizing complex, open-ended answers involving multi-step reasoning and LaTeX-formatted expressions over simple multiple-choice questions; and (4) Rigorous Quality Control, employing a multi-stage process of expert peer review and adversarial testing to ensure question difficulty, scientific value, and correctness. We also propose a robust evaluation paradigm using a panel of LLM judges for automated, nuanced assessment of complex answers. Preliminary results on leading models demonstrate ATLAS's effectiveness in differentiating their advanced scientific reasoning capabilities. We plan to develop ATLAS into a long-term, open, community-driven platform to provide a reliable "ruler" for progress toward Artificial General Intelligence.
691d30d20247f3d258ef31e0
Hongwei Liu
691d30d20247f3d258ef31e1
Junnan Liu
claimed_verified
2025-11-20T08:49:59.427Z
643d26979347842571bc9613
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/3heFf7h3jbhhJWJ4JfGfh.jpeg
Junnan Liu
jnanliu
user
jnanliu
691d30d20247f3d258ef31e2
Shudong Liu
claimed_verified
2025-11-19T08:39:19.734Z
654ce87af0b05673196a9f45
/avatars/7b9c854eb98e487e3057479b1c7860ac.svg
Shudong Liu
Sudanl
user
Sudanl
691d30d20247f3d258ef31e3
Haodong Duan
691d30d20247f3d258ef31e4
Yuqiang Li
691d30d20247f3d258ef31e5
Mao Su
691d30d20247f3d258ef31e6
Xiaohong Liu
691d30d20247f3d258ef31e7
Guangtao Zhai
691d30d20247f3d258ef31e8
Xinyu Fang
claimed_verified
2025-12-05T08:39:14.099Z
64f5f8dd9b17cd59c453c57f
https://cdn-avatars.huggingface.co/v1/production/uploads/64f5f8dd9b17cd59c453c57f/MulhwLcePFUWUQel8LQZ8.jpeg
Xinyu Fang
nebulae09
user
nebulae09
691d30d20247f3d258ef31e9
Qianhong Ma
691d30d20247f3d258ef31ea
Taolin Zhang
691d30d20247f3d258ef31eb
Zihan Ma
claimed_verified
2025-11-20T08:50:01.178Z
677e869467f3bb8d8215eec6
https://cdn-avatars.huggingface.co/v1/production/uploads/677e869467f3bb8d8215eec6/kEC6JOKObgLHA22jRcP4H.jpeg
Zihan Ma
MichaelErchi
user
MichaelErchi
691d30d20247f3d258ef31ec
Yufeng Zhao
691d30d20247f3d258ef31ed
Peiheng Zhou
691d30d20247f3d258ef31ee
Linchen Xiao
691d30d20247f3d258ef31ef
Wenlong Zhang
691d30d20247f3d258ef31f0
Shijie Zhou
691d30d20247f3d258ef31f1
Xingjian Ma
691d30d20247f3d258ef31f2
Siqi Sun
691d30d20247f3d258ef31f3
Jiaye Ge
691d30d20247f3d258ef31f4
Meng Li
691d30d20247f3d258ef31f5
Yuhong Liu
691d30d20247f3d258ef31f6
Jianxin Dong
691d30d20247f3d258ef31f7
Jiaying Li
691d30d20247f3d258ef31f8
Hui Wu
691d30d20247f3d258ef31f9
Hanwen Liang
691d30d20247f3d258ef31fa
Jintai Lin
691d30d20247f3d258ef31fb
Yanting Wang
691d30d20247f3d258ef31fc
Jie Dong
691d30d20247f3d258ef31fd
Tong Zhu
691d30d20247f3d258ef31fe
Tianfan Fu
691d30d20247f3d258ef31ff
Conghui He
691d30d20247f3d258ef3200
Qi Zhang
691d30d20247f3d258ef3201
Songyang Zhang
691d30d20247f3d258ef3202
Lei Bai
691d30d20247f3d258ef3203
Kai Chen
ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning
The rapid advancement of Large Language Models (LLMs) has led to performance saturation on many established benchmarks, questioning their ability to distinguish frontier models. Concurrently, existing high-difficulty benchmarks often suffer from narrow disciplinary focus, oversimplified answer formats, and vulnerability to data contamination, creating a fidelity gap with real-world scientific inquiry. To address these challenges, we introduce ATLAS (AGI-Oriented Testbed for Logical Application in Science), a large-scale, high-difficulty, and cross-disciplinary evaluation suite composed of approximately 800 original problems. Developed by domain experts (PhD-level and above), ATLAS spans seven core scientific fields: mathematics, physics, chemistry, biology, computer science, earth science, and materials science. Its key features include: (1) High Originality and Contamination Resistance, with all questions newly created or substantially adapted to prevent test data leakage; (2) Cross-Disciplinary Focus, designed to assess models' ability to integrate knowledge and reason across scientific domains; (3) High-Fidelity Answers, prioritizing complex, open-ended answers involving multi-step reasoning and LaTeX-formatted expressions over simple multiple-choice questions; and (4) Rigorous Quality Control, employing a multi-stage process of expert peer review and adversarial testing to ensure question difficulty, scientific value, and correctness. We also propose a robust evaluation paradigm using a panel of LLM judges for automated, nuanced assessment of complex answers. Preliminary results on leading models demonstrate ATLAS's effectiveness in differentiating their advanced scientific reasoning capabilities. We plan to develop ATLAS into a long-term, open, community-driven platform to provide a reliable "ruler" for progress toward Artificial General Intelligence.
ATLAS, a large-scale, cross-disciplinary evaluation suite, addresses the limitations of existing benchmarks by providing high-difficulty, original, and high-fidelity scientific problems to assess the reasoning capabilities of Large Language Models.
Large Language Models LLMs ATLAS AGI-Oriented Testbed logical application scientific inquiry domain experts multi-disciplinary multi-step reasoning LaTeX-formatted expressions LLM judges adversarial testing expert peer review