General Agent Evaluation
The promise of general-purpose agents - systems that perform tasks in unfamiliar environments without domain-specific engineering - remains largely unrealized. Existing agents are predominantly specialized, and while emerging implementations like OpenAI SDK Agent and Claude Code hint at broader capabilities, no systematic evaluation of their general performance has been pursued. Current agentic benchmarks assume domain-specific integration, encoding task information in ways that preclude fair evaluation of general agents. This paper frames general-agent evaluation as a first-class research objective. We propose conceptual principles for such evaluation, a Unified Protocol enabling agent-benchmark integration, and Exgentic - a practical framework for general agent evaluation. We benchmark five prominent agent implementations across six environments as the first Open General Agent Leaderboard. Our experiments show that general agents generalize across diverse environments, achieving performance comparable to domain-specific agents without any environment-specific tuning. We release our evaluation protocol, framework, and leaderboard to establish a foundation for systematic research on general-purpose agents.
69a1c790bc2d66e68888eff6
Elron Bandel
claimed_verified
2026-03-02T09:39:04.274Z
5fc0292de45c5468456e022b
https://cdn-avatars.huggingface.co/v1/production/uploads/5fc0292de45c5468456e022b/MEuHFEf6LtTbcptc0heoM.png
Elron Bandel
Elron
user
Elron
69a1c790bc2d66e68888eff7
Asaf Yehudai
69a1c790bc2d66e68888eff8
Lilach Eden
69a1c790bc2d66e68888eff9
Yehoshua Sagron
69a1c790bc2d66e68888effa
Yotam Perlitz
69a1c790bc2d66e68888effb
Elad Venezian
69a1c790bc2d66e68888effc
Natalia Razinkov
69a1c790bc2d66e68888effd
Natan Ergas
69a1c790bc2d66e68888effe
Shlomit Shachor Ifergan
69a1c790bc2d66e68888efff
Segev Shlomov
claimed_verified
2026-03-03T10:42:54.582Z
648aad9d4b0743d63f8b4c7f
/avatars/a3fe876607d7659f232c4a67ec26c86c.svg
Segev Shlomov
segevshlomov
user
segevshlomov
69a1c790bc2d66e68888f000
Michal Jacovi
69a1c790bc2d66e68888f001
Leshem Choshen
69a1c790bc2d66e68888f002
Liat Ein-Dor
69a1c790bc2d66e68888f003
Yoav Katz
69a1c790bc2d66e68888f004
Michal Shmueli-Scheuer
General Agent Evaluation
The promise of general-purpose agents - systems that perform tasks in unfamiliar environments without domain-specific engineering - remains largely unrealized. Existing agents are predominantly specialized, and while emerging implementations like OpenAI SDK Agent and Claude Code hint at broader capabilities, no systematic evaluation of their general performance has been pursued. Current agentic benchmarks assume domain-specific integration, encoding task information in ways that preclude fair evaluation of general agents. This paper frames general-agent evaluation as a first-class research objective. We propose conceptual principles for such evaluation, a Unified Protocol enabling agent-benchmark integration, and Exgentic - a practical framework for general agent evaluation. We benchmark five prominent agent implementations across six environments as the first Open General Agent Leaderboard. Our experiments show that general agents generalize across diverse environments, achieving performance comparable to domain-specific agents without any environment-specific tuning. We release our evaluation protocol, framework, and leaderboard to establish a foundation for systematic research on general-purpose agents.
General-purpose agents remain underdeveloped despite promising implementations, necessitating systematic evaluation frameworks and benchmarks to assess their true versatility across diverse environments.
general-purpose agents agentic benchmarks unified protocol Exgentic framework Open General Agent Leaderboard