Hunt Instead of Wait: Evaluating Deep Data Research on Large Language Models
The agency expected of Agentic Large Language Models goes beyond answering correctly, requiring autonomy to set goals and decide what to explore. We term this investigatory intelligence, distinguishing it from executional intelligence, which merely completes assigned tasks. Data Science provides a natural testbed, as real-world analysis starts from raw data rather than explicit queries, yet few benchmarks focus on it. To address this, we introduce Deep Data Research (DDR), an open-ended task where LLMs autonomously extract key insights from databases, and DDR-Bench, a large-scale, checklist-based benchmark that enables verifiable evaluation. Results show that while frontier models display emerging agency, long-horizon exploration remains challenging. Our analysis highlights that effective investigatory intelligence depends not only on agent scaffolding or merely scaling, but also on intrinsic strategies of agentic models.
6981c6f9ce18b18628096535
Wei Liu
claimed_verified
2026-02-04T12:31:42.414Z
66e2932e5c100c12aa2def39
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/FiQ5Fap-qVqnXeULGPYs6.png
weiliu
thinkwee
user
thinkwee
6981c6f9ce18b18628096536
Peijie Yu
claimed_verified
2026-02-05T10:55:28.024Z
62d0e8220962b7b62fc47bb5
/avatars/e9b49734d4ccbda84db41b93d507a1fc.svg
jpy
jpy
user
jpy
6981c6f9ce18b18628096537
Michele Orini
6981c6f9ce18b18628096538
Yali Du
6981c6f9ce18b18628096539
Yulan He
Hunt Instead of Wait: Evaluating Deep Data Research on Large Language Models
The agency expected of Agentic Large Language Models goes beyond answering correctly, requiring autonomy to set goals and decide what to explore. We term this investigatory intelligence, distinguishing it from executional intelligence, which merely completes assigned tasks. Data Science provides a natural testbed, as real-world analysis starts from raw data rather than explicit queries, yet few benchmarks focus on it. To address this, we introduce Deep Data Research (DDR), an open-ended task where LLMs autonomously extract key insights from databases, and DDR-Bench, a large-scale, checklist-based benchmark that enables verifiable evaluation. Results show that while frontier models display emerging agency, long-horizon exploration remains challenging. Our analysis highlights that effective investigatory intelligence depends not only on agent scaffolding or merely scaling, but also on intrinsic strategies of agentic models.
Agentic large language models require investigatory intelligence for autonomous data analysis, demonstrated through the Deep Data Research benchmark that evaluates their ability to extract insights from databases without explicit queries.
agentic large language models investigatory intelligence executional intelligence data science open-ended task database analysis benchmark evaluation autonomous agents long-horizon exploration intrinsic strategies