Failing to Explore: Language Models on Interactive Tasks
We evaluate language models on their ability to explore interactive environments under a limited interaction budget. We introduce three parametric tasks with controllable exploration difficulty, spanning continuous and discrete environments. Across state-of-the-art models, we find systematic under-exploration and suboptimal solutions, with performance often significantly worse than simple explore--exploit heuristic baselines and scaling weakly as the budget increases. Finally, we study two lightweight interventions: splitting a fixed budget into parallel executions, which surprisingly improves performance despite a no-gain theoretical result for our tasks, and periodically summarizing the interaction history, which preserves key discoveries and further improves exploration.
698504344ad556f294b7ea38
Mahdi JafariRaviz
claimed_verified
2026-02-06T18:52:32.904Z
68a0e04b4fb0c20163c64066
/avatars/e12137e46d3965e708d1ecefebf57d20.svg
Mahdi JafariRaviz
AghaTizi
user
AghaTizi
698504344ad556f294b7ea39
Keivan Rezaei
698504344ad556f294b7ea3a
Arshia Soltani Moakhar
claimed_verified
2026-02-06T18:52:27.622Z
6304de2b34c824b1724f3bb2
/avatars/e1aede9aba33476736f6a7e45a748e69.svg
Arshia Soltani Moakhar
ckodser
user
ckodser
698504344ad556f294b7ea3b
Zahra Sodagar
698504344ad556f294b7ea3c
Yize Cheng
claimed_verified
2026-02-06T18:52:30.468Z
66fa2c61c25c3fcb32f9f131
/avatars/05387c30f2d1803fa0a5b176c3706772.svg
Yize Cheng
yizecheng
user
yizecheng
698504344ad556f294b7ea3d
Soheil Feizi
Failing to Explore: Language Models on Interactive Tasks
We evaluate language models on their ability to explore interactive environments under a limited interaction budget. We introduce three parametric tasks with controllable exploration difficulty, spanning continuous and discrete environments. Across state-of-the-art models, we find systematic under-exploration and suboptimal solutions, with performance often significantly worse than simple explore--exploit heuristic baselines and scaling weakly as the budget increases. Finally, we study two lightweight interventions: splitting a fixed budget into parallel executions, which surprisingly improves performance despite a no-gain theoretical result for our tasks, and periodically summarizing the interaction history, which preserves key discoveries and further improves exploration.
Language models exhibit limited exploration capabilities in interactive environments, with performance improvements achieved through budget allocation strategies and historical summarization techniques.
language models interactive environments exploration difficulty parametric tasks exploration budget heuristic baselines parallel executions interaction history summarization