Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
Despite advances in scientific AI, a coherent framework for Scientific General Intelligence (SGI)-the ability to autonomously conceive, investigate, and reason across scientific domains-remains lacking. We present an operational SGI definition grounded in the Practical Inquiry Model (PIM: Deliberation, Conception, Action, Perception) and operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning. SGI-Bench comprises over 1,000 expert-curated, cross-disciplinary samples inspired by Science's 125 Big Questions, enabling systematic evaluation of state-of-the-art LLMs. Results reveal gaps: low exact match (10--20%) in deep research despite step-level alignment; ideas lacking feasibility and detail; high code executability but low execution result accuracy in dry experiments; low sequence fidelity in wet protocols; and persistent multimodal comparative-reasoning challenges. We further introduce Test-Time Reinforcement Learning (TTRL), which optimizes retrieval-augmented novelty rewards at inference, enhancing hypothesis novelty without reference answer. Together, our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
6948b09934f46eaf46cbb214
Wanghan Xu
claimed_verified
2025-12-22T10:58:47.069Z
65f3f43fc9940817ca9a427b
https://cdn-avatars.huggingface.co/v1/production/uploads/65f3f43fc9940817ca9a427b/02NN3XjSsbgWDhjrJWtVL.jpeg
Wanghan Xu
CoCoOne
user
CoCoOne
6948b09934f46eaf46cbb215
Yuhao Zhou
claimed_verified
2025-12-25T20:52:06.289Z
63bab9c1bb6a2fabd14421bd
/avatars/c47030cf167072bf6ce3421f025c7746.svg
Yuhao Zhou
Soptq
user
Soptq
6948b09934f46eaf46cbb216
Yifan Zhou
claimed_verified
2025-12-25T20:52:08.888Z
659d2dff20cf0b934bbee513
https://cdn-avatars.huggingface.co/v1/production/uploads/659d2dff20cf0b934bbee513/9e9R852Zr2R82h64eUUQl.jpeg
Yifan Zhou
yingmanji
user
yingmanji
6948b09934f46eaf46cbb217
Qinglong Cao
6948b09934f46eaf46cbb218
Shuo Li
6948b09934f46eaf46cbb219
Jia Bu
6948b09934f46eaf46cbb21a
Bo Liu
claimed_verified
2025-12-22T10:58:41.476Z
61e6dd8a82b19b93e1a51fa6
/avatars/babbee52793a35dd5754d000946dd1ee.svg
Kelvin Liu
BoKelvin
user
BoKelvin
6948b09934f46eaf46cbb21b
Yixin Chen
6948b09934f46eaf46cbb21c
Xuming He
6948b09934f46eaf46cbb21d
Xiangyu Zhao
6948b09934f46eaf46cbb21e
Xiang Zhuang
6948b09934f46eaf46cbb21f
Fengxiang Wang
6948b09934f46eaf46cbb220
Zhiwang Zhou
6948b09934f46eaf46cbb221
Qiantai Feng
6948b09934f46eaf46cbb222
Wenxuan Huang
claimed_verified
2026-02-13T09:38:23.491Z
675118b088a927f8898f81b4
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/qVVgc2kuR37QqO5-Lu8Xq.png
Wilson Huang
WilsonHwang
user
WilsonHwang
6948b09934f46eaf46cbb223
Jiaqi Wei
claimed_verified
2025-12-22T10:58:43.408Z
6539bc7756c9b35961021fa8
/avatars/b0140589c0a435c903c93d93a1a6ee8b.svg
Jiaqi Wei
VitaCoco
user
VitaCoco
6948b09934f46eaf46cbb224
Hao Wu
6948b09934f46eaf46cbb225
Yuejin Yang
6948b09934f46eaf46cbb226
Guangshuai Wang
6948b09934f46eaf46cbb227
Sheng Xu
6948b09934f46eaf46cbb228
Ziyan Huang
6948b09934f46eaf46cbb229
Xinyao Liu
6948b09934f46eaf46cbb22a
Jiyao Liu
6948b09934f46eaf46cbb22b
Cheng Tang
6948b09934f46eaf46cbb22c
Wei Li
6948b09934f46eaf46cbb22d
Ying Chen
6948b09934f46eaf46cbb22e
Junzhi Ning
6948b09934f46eaf46cbb22f
Pengfei Jiang
6948b09934f46eaf46cbb230
Chenglong Ma
6948b09934f46eaf46cbb231
Ye Du
6948b09934f46eaf46cbb232
Changkai Ji
6948b09934f46eaf46cbb233
Huihui Xu
6948b09934f46eaf46cbb234
Ming Hu
6948b09934f46eaf46cbb235
Jiangbin Zheng
6948b09934f46eaf46cbb236
Xin Chen
6948b09934f46eaf46cbb237
Yucheng Wu
6948b09934f46eaf46cbb238
Feifei Jiang
6948b09934f46eaf46cbb239
Xi Chen
6948b09934f46eaf46cbb23a
Xiangru Tang
6948b09934f46eaf46cbb23b
Yuchen Fu
6948b09934f46eaf46cbb23c
Yingzhou Lu
6948b09934f46eaf46cbb23d
Yuanyuan Zhang
6948b09934f46eaf46cbb23e
Lihao Sun
6948b09934f46eaf46cbb23f
Chengbo Li
6948b09934f46eaf46cbb240
Jinzhe Ma
6948b09934f46eaf46cbb241
Wanhao Liu
6948b09934f46eaf46cbb242
Yating Liu
6948b09934f46eaf46cbb243
Kuo-Cheng Wu
6948b09934f46eaf46cbb244
Shengdu Chai
6948b09934f46eaf46cbb245
Yizhou Wang
6948b09934f46eaf46cbb246
Ouwen Zhangjin
6948b09934f46eaf46cbb247
Chen Tang
6948b09934f46eaf46cbb248
Shufei Zhang
6948b09934f46eaf46cbb249
Wenbo Cao
6948b09934f46eaf46cbb24a
Junjie Ren
6948b09934f46eaf46cbb24b
Taoyong Cui
6948b09934f46eaf46cbb24c
Zhouheng Yao
6948b09934f46eaf46cbb24d
Juntao Deng
6948b09934f46eaf46cbb24e
Yijie Sun
6948b09934f46eaf46cbb24f
Feng Liu
6948b09934f46eaf46cbb250
Wangxu Wei
6948b09934f46eaf46cbb251
Jingyi Xu
6948b09934f46eaf46cbb252
Zhangrui Li
6948b09934f46eaf46cbb253
Junchao Gong
6948b09934f46eaf46cbb254
Zijie Guo
6948b09934f46eaf46cbb255
Zhiyu Yao
6948b09934f46eaf46cbb256
Zaoyu Chen
6948b09934f46eaf46cbb257
Tianhao Peng
6948b09934f46eaf46cbb258
Fangchen Yu
claimed_verified
2025-12-22T10:58:45.323Z
68ad9cb3bcaa8d84217a8bdf
/avatars/dbb3199cf5bfc2acdbd38069c823c027.svg
Fangchen Yu
SciYu
user
SciYu
6948b09934f46eaf46cbb259
Bo Zhang
6948b09934f46eaf46cbb25a
Dongzhan Zhou
6948b09934f46eaf46cbb25b
Shixiang Tang
6948b09934f46eaf46cbb25c
Jiaheng Liu
6948b09934f46eaf46cbb25d
Fenghua Ling
6948b09934f46eaf46cbb25e
Yan Lu
6948b09934f46eaf46cbb25f
Yuchen Ren
6948b09934f46eaf46cbb260
Ben Fei
6948b09934f46eaf46cbb261
Zhen Zhao
6948b09934f46eaf46cbb262
Xinyu Gu
6948b09934f46eaf46cbb263
Rui Su
6948b09934f46eaf46cbb264
Xiao-Ming Wu
6948b09934f46eaf46cbb265
Weikang Si
6948b09934f46eaf46cbb266
Yang Liu
6948b09934f46eaf46cbb267
Hao Chen
6948b09934f46eaf46cbb268
Xiangchao Yan
6948b09934f46eaf46cbb269
Xue Yang
6948b09934f46eaf46cbb26a
Junchi Yan
6948b09934f46eaf46cbb26b
Jiamin Wu
6948b09934f46eaf46cbb26c
Qihao Zheng
6948b09934f46eaf46cbb26d
Chenhui Li
6948b09934f46eaf46cbb26e
Zhiqiang Gao
6948b09934f46eaf46cbb26f
Hao Kong
6948b09934f46eaf46cbb270
Junjun He
6948b09934f46eaf46cbb271
Mao Su
6948b09934f46eaf46cbb272
Tianfan Fu
6948b09934f46eaf46cbb273
Peng Ye
6948b09934f46eaf46cbb274
Chunfeng Song
6948b09934f46eaf46cbb275
Nanqing Dong
6948b09934f46eaf46cbb276
Yuqiang Li
6948b09934f46eaf46cbb277
Huazhu Fu
6948b09934f46eaf46cbb278
Siqi Sun
6948b09934f46eaf46cbb279
Lijing Cheng
6948b09934f46eaf46cbb27a
Jintai Lin
6948b09934f46eaf46cbb27b
Wanli Ouyang
6948b09934f46eaf46cbb27c
Bowen Zhou
6948b09934f46eaf46cbb27d
Wenlong Zhang
6948b09934f46eaf46cbb27e
Lei Bai
Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
Despite advances in scientific AI, a coherent framework for Scientific General Intelligence (SGI)-the ability to autonomously conceive, investigate, and reason across scientific domains-remains lacking. We present an operational SGI definition grounded in the Practical Inquiry Model (PIM: Deliberation, Conception, Action, Perception) and operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning. SGI-Bench comprises over 1,000 expert-curated, cross-disciplinary samples inspired by Science's 125 Big Questions, enabling systematic evaluation of state-of-the-art LLMs. Results reveal gaps: low exact match (10--20%) in deep research despite step-level alignment; ideas lacking feasibility and detail; high code executability but low execution result accuracy in dry experiments; low sequence fidelity in wet protocols; and persistent multimodal comparative-reasoning challenges. We further introduce Test-Time Reinforcement Learning (TTRL), which optimizes retrieval-augmented novelty rewards at inference, enhancing hypothesis novelty without reference answer. Together, our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
A framework for Scientific General Intelligence (SGI) is presented, evaluated using SGI-Bench, and improved with Test-Time Reinforcement Learning, highlighting gaps in existing models' scientific capabilities.
Scientific General Intelligence SGI Practical Inquiry Model PIM deep research idea generation dry experiments wet experiments experimental reasoning SGI-Bench Big Questions Low exact match feasibility detail code executability execution result accuracy sequence fidelity multimodal comparative-reasoning Test-Time Reinforcement Learning TTRL retrieval-augmented novelty rewards hypothesis novelty