EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies
Long-horizon planning is widely recognized as a core capability of autonomous LLM-based agents; however, current evaluation frameworks suffer from being largely episodic, domain-specific, or insufficiently grounded in persistent economic dynamics. We introduce EcoGym, a generalizable benchmark for continuous plan-and-execute decision making in interactive economies. EcoGym comprises three diverse environments: Vending, Freelance, and Operation, implemented in a unified decision-making process with standardized interfaces, and budgeted actions over an effectively unbounded horizon (1000+ steps if 365 day-loops for evaluation). The evaluation of EcoGym is based on business-relevant outcomes (e.g., net worth, income, and DAU), targeting long-term strategic coherence and robustness under partial observability and stochasticity. Experiments across eleven leading LLMs expose a systematic tension: no single model dominates across all three scenarios. Critically, we find that models exhibit significant suboptimality in either high-level strategies or efficient actions executions. EcoGym is released as an open, extensible testbed for transparent long-horizon agent evaluation and for studying controllability-utility trade-offs in realistic economic settings.
698d873b65c0d15a6d1622b9
Xavier Hu
698d873b65c0d15a6d1622ba
Jinxiang Xia
698d873b65c0d15a6d1622bb
Shengze Xu
698d873b65c0d15a6d1622bc
Kangqi Song
698d873b65c0d15a6d1622bd
Yishuo Yuan
698d873b65c0d15a6d1622be
Guibin Zhang
698d873b65c0d15a6d1622bf
JinCheng Ren
698d873b65c0d15a6d1622c0
Boyu Feng
698d873b65c0d15a6d1622c1
Li Lu
698d873b65c0d15a6d1622c2
Tieyong Zeng
698d873b65c0d15a6d1622c3
Jiaheng Liu
698d873b65c0d15a6d1622c4
Minghao Liu
claimed_verified
2026-02-25T17:32:47.937Z
6417d9ea8f689506e7148417
https://cdn-avatars.huggingface.co/v1/production/uploads/6417d9ea8f689506e7148417/bAYcruWNw4WvmuQcGgcwC.jpeg
minghao
Liam-Liu
user
Liam-Liu
698d873b65c0d15a6d1622c5
He Zhu
698d873b65c0d15a6d1622c6
Yuchen Eleanor Jiang
698d873b65c0d15a6d1622c7
Wei Wang
698d873b65c0d15a6d1622c8
Wangchunshu Zhou
EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies
Long-horizon planning is widely recognized as a core capability of autonomous LLM-based agents; however, current evaluation frameworks suffer from being largely episodic, domain-specific, or insufficiently grounded in persistent economic dynamics. We introduce EcoGym, a generalizable benchmark for continuous plan-and-execute decision making in interactive economies. EcoGym comprises three diverse environments: Vending, Freelance, and Operation, implemented in a unified decision-making process with standardized interfaces, and budgeted actions over an effectively unbounded horizon (1000+ steps if 365 day-loops for evaluation). The evaluation of EcoGym is based on business-relevant outcomes (e.g., net worth, income, and DAU), targeting long-term strategic coherence and robustness under partial observability and stochasticity. Experiments across eleven leading LLMs expose a systematic tension: no single model dominates across all three scenarios. Critically, we find that models exhibit significant suboptimality in either high-level strategies or efficient actions executions. EcoGym is released as an open, extensible testbed for transparent long-horizon agent evaluation and for studying controllability-utility trade-offs in realistic economic settings.
EcoGym presents a generalizable benchmark for evaluating long-horizon planning capabilities of LLM-based agents in interactive economic environments with persistent dynamics and multi-scenario evaluation.
long-horizon planning LLM-based agents continuous plan-and-execute decision making interactive economies persistent economic dynamics business-relevant outcomes partial observability stochasticity controllability-utility trade-offs