The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios
The rapid evolution of Multi-modal Large Language Models (MLLMs) has advanced workflow automation; however, existing research mainly targets performance upper bounds in static environments, overlooking robustness for stochastic real-world deployment. We identify three key challenges: dynamic task scheduling, active exploration under uncertainty, and continuous learning from experience. To bridge this gap, we introduce , a dynamic evaluation environment that simulates a "trainee" agent continuously exploring a novel setting. Unlike traditional benchmarks,  evaluates agents along three dimensions: (1) context-aware scheduling for streaming tasks with varying priorities; (2) prudent information acquisition to reduce hallucination via active exploration; and (3) continuous evolution by distilling generalized strategies from rule-based, dynamically generated tasks. Experiments show that cutting-edge agents have significant deficiencies in dynamic environments, especially in active exploration and continual learning. Our work establishes a framework for assessing agent reliability, shifting evaluation from static tests to realistic, production-oriented scenarios. Our codes are available at https://github.com/KnowledgeXLab/EvoEnv
69683d0b0ac10a06522f698e
Daocheng Fu
69683d0b0ac10a06522f698f
Jianbiao Mei
claimed_verified
2026-01-15T15:03:34.293Z
65827953d73d6402f7ae332d
/avatars/d57eae62171ece7a237eaf4ea938a31c.svg
Mei
Jianbiao
user
Jianbiao
69683d0b0ac10a06522f6990
Rong Wu
claimed_verified
2026-01-19T09:24:35.806Z
66fa9fa56b620c614cdb1b57
/avatars/907c101ba4528de05d2ce4bb1ad82005.svg
Rong Wu
Edaizi
user
Edaizi
69683d0b0ac10a06522f6991
Xuemeng Yang
69683d0b0ac10a06522f6992
Jia Xu
69683d0b0ac10a06522f6993
Ding Wang
69683d0b0ac10a06522f6994
Pinlong Cai
69683d0b0ac10a06522f6995
Yong Liu
69683d0b0ac10a06522f6996
Licheng Wen
69683d0b0ac10a06522f6997
Botian Shi
The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios
The rapid evolution of Multi-modal Large Language Models (MLLMs) has advanced workflow automation; however, existing research mainly targets performance upper bounds in static environments, overlooking robustness for stochastic real-world deployment. We identify three key challenges: dynamic task scheduling, active exploration under uncertainty, and continuous learning from experience. To bridge this gap, we introduce , a dynamic evaluation environment that simulates a "trainee" agent continuously exploring a novel setting. Unlike traditional benchmarks,  evaluates agents along three dimensions: (1) context-aware scheduling for streaming tasks with varying priorities; (2) prudent information acquisition to reduce hallucination via active exploration; and (3) continuous evolution by distilling generalized strategies from rule-based, dynamically generated tasks. Experiments show that cutting-edge agents have significant deficiencies in dynamic environments, especially in active exploration and continual learning. Our work establishes a framework for assessing agent reliability, shifting evaluation from static tests to realistic, production-oriented scenarios. Our codes are available at https://github.com/KnowledgeXLab/EvoEnv
Dynamic evaluation environment for MLLMs addressing robustness challenges in real-world deployment through context-aware scheduling, active exploration, and continuous learning.
Multi-modal Large Language Models dynamic task scheduling active exploration continuous learning context-aware scheduling prudent information acquisition generalized strategies rule-based tasks production-oriented scenarios