MemOCR: Layout-Aware Visual Memory for Efficient Long-Horizon Reasoning
Long-horizon agentic reasoning necessitates effectively compressing growing interaction histories into a limited context window. Most existing memory systems serialize history as text, where token-level cost is uniform and scales linearly with length, often spending scarce budget on low-value details. To this end, we introduce MemOCR, a multimodal memory agent that improves long-horizon reasoning under tight context budgets by allocating memory space with adaptive information density through visual layout. Concretely, MemOCR maintains a structured rich-text memory (e.g., headings, highlights) and renders it into an image that the agent consults for memory access, visually prioritizing crucial evidence while aggressively compressing auxiliary details. To ensure robustness across varying memory budgets, we train MemOCR with reinforcement learning under budget-aware objectives that expose the agent to diverse compression levels. Across long-context multi-hop and single-hop question-answering benchmarks, MemOCR outperforms strong text-based baselines and achieves more effective context utilization under extreme budgets.
697c7da1a67238fac88cc2d0
Yaorui Shi
claimed_verified
2026-02-04T12:33:57.232Z
63edd2d1f765928ceeb49057
https://cdn-avatars.huggingface.co/v1/production/uploads/1676530369930-noauth.png
Yaorui SHI
yrshi
user
yrshi
697c7da1a67238fac88cc2d1
Shugui Liu
697c7da1a67238fac88cc2d2
Yu Yang
697c7da1a67238fac88cc2d3
Wenyu Mao
697c7da1a67238fac88cc2d4
Yuxin Chen
697c7da1a67238fac88cc2d5
Qi GU
697c7da1a67238fac88cc2d6
Hui Su
697c7da1a67238fac88cc2d7
Xunliang Cai
697c7da1a67238fac88cc2d8
Xiang Wang
697c7da1a67238fac88cc2d9
An Zhang
MemOCR: Layout-Aware Visual Memory for Efficient Long-Horizon Reasoning
Long-horizon agentic reasoning necessitates effectively compressing growing interaction histories into a limited context window. Most existing memory systems serialize history as text, where token-level cost is uniform and scales linearly with length, often spending scarce budget on low-value details. To this end, we introduce MemOCR, a multimodal memory agent that improves long-horizon reasoning under tight context budgets by allocating memory space with adaptive information density through visual layout. Concretely, MemOCR maintains a structured rich-text memory (e.g., headings, highlights) and renders it into an image that the agent consults for memory access, visually prioritizing crucial evidence while aggressively compressing auxiliary details. To ensure robustness across varying memory budgets, we train MemOCR with reinforcement learning under budget-aware objectives that expose the agent to diverse compression levels. Across long-context multi-hop and single-hop question-answering benchmarks, MemOCR outperforms strong text-based baselines and achieves more effective context utilization under extreme budgets.
MemOCR is a multimodal memory agent that enhances long-horizon reasoning by adaptively compressing interaction histories into visual layouts, enabling efficient context utilization under tight budget constraints.
memory systems context window visual layout structured rich-text memory reinforcement learning context utilization long-horizon reasoning