RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipulating objects that become temporarily occluded. Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms; however, their evaluations remain confined to narrow, non-standardized settings. This limits their systematic understanding, comparison, and progress measurement. To address these challenges, we introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models in long-horizon, history-dependent scenarios. Our benchmark comprises 16 manipulation tasks constructed under a carefully designed taxonomy that evaluates temporal, spatial, object, and procedural memory. We further develop a suite of 14 memory-augmented VLA variants built on the π0.5 backbone to systematically explore different memory representations across multiple integration strategies. Experimental results show that the effectiveness of memory representations is highly task-dependent, with each design offering distinct advantages and limitations across different tasks. Videos and code can be found at our website https://robomme.github.io.
69add8d29893f58272f7400b
Yinpei Dai
claimed_verified
2026-03-09T08:56:18.798Z
648b889575d21f74acf9cbb1
/avatars/0ffee0809f9e2670821721c005d8dde4.svg
Dai
Yinpei
user
Yinpei
69add8d29893f58272f7400c
Hongze Fu
69add8d29893f58272f7400d
Jayjun Lee
69add8d29893f58272f7400e
Yuejiang Liu
69add8d29893f58272f7400f
Haoran Zhang
69add8d29893f58272f74010
Jianing Yang
69add8d29893f58272f74011
Chelsea Finn
69add8d29893f58272f74012
Nima Fazeli
69add8d29893f58272f74013
Joyce Chai
RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipulating objects that become temporarily occluded. Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms; however, their evaluations remain confined to narrow, non-standardized settings. This limits their systematic understanding, comparison, and progress measurement. To address these challenges, we introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models in long-horizon, history-dependent scenarios. Our benchmark comprises 16 manipulation tasks constructed under a carefully designed taxonomy that evaluates temporal, spatial, object, and procedural memory. We further develop a suite of 14 memory-augmented VLA variants built on the π0.5 backbone to systematically explore different memory representations across multiple integration strategies. Experimental results show that the effectiveness of memory representations is highly task-dependent, with each design offering distinct advantages and limitations across different tasks. Videos and code can be found at our website https://robomme.github.io.
RoboMME presents a large-scale standardized benchmark for evaluating vision-language-action models in long-horizon robotic manipulation tasks requiring memory mechanisms.
vision-language-action models memory mechanisms long-horizon tasks history-dependent scenarios standardized benchmark temporal memory spatial memory object memory procedural memory memory-augmented VLA variants π0.5 backbone