Why LLMs Aren't Scientists Yet: Lessons from Four Autonomous Research Attempts
We report a case study of four end-to-end attempts to autonomously generate ML research papers using a pipeline of six LLM agents mapped to stages of the scientific workflow. Of these four, three attempts failed during implementation or evaluation. One completed the pipeline and was accepted to Agents4Science 2025, an experimental inaugural venue that required AI systems as first authors, passing both human and multi-AI review. From these attempts, we document six recurring failure modes: bias toward training data defaults, implementation drift under execution pressure, memory and context degradation across long-horizon tasks, overexcitement that declares success despite obvious failures, insufficient domain intelligence, and weak scientific taste in experimental design. We conclude by discussing four design principles for more robust AI-scientist systems, implications for autonomous scientific discovery, and we release all prompts, artifacts, and outputs at https://github.com/Lossfunk/ai-scientist-artefacts-v1
695f9e182450f142afb3c498
Dhruv Trehan
claimed_verified
2026-01-09T08:36:05.134Z
66261924d78c254961346767
/avatars/7aa7e14db3eb682caa4ca9fd7cd72add.svg
Dhruv Trehan
DhruvTre
user
DhruvTre
695f9e182450f142afb3c499
Paras Chopra
Why LLMs Aren't Scientists Yet: Lessons from Four Autonomous Research Attempts
We report a case study of four end-to-end attempts to autonomously generate ML research papers using a pipeline of six LLM agents mapped to stages of the scientific workflow. Of these four, three attempts failed during implementation or evaluation. One completed the pipeline and was accepted to Agents4Science 2025, an experimental inaugural venue that required AI systems as first authors, passing both human and multi-AI review. From these attempts, we document six recurring failure modes: bias toward training data defaults, implementation drift under execution pressure, memory and context degradation across long-horizon tasks, overexcitement that declares success despite obvious failures, insufficient domain intelligence, and weak scientific taste in experimental design. We conclude by discussing four design principles for more robust AI-scientist systems, implications for autonomous scientific discovery, and we release all prompts, artifacts, and outputs at https://github.com/Lossfunk/ai-scientist-artefacts-v1
A case study of four attempts to autonomously generate ML research papers using LLM agents reveals recurring failure modes and proposes design principles for robust AI-scientist systems.
LLM agents scientific workflow autonomous scientific discovery AI-scientist systems