GEMS: Agent-Native Multimodal Generation with Memory and Skills
Recent multimodal generation models have achieved remarkable progress on general-purpose generation tasks, yet continue to struggle with complex instructions and specialized downstream tasks. Inspired by the success of advanced agent frameworks such as Claude Code, we propose GEMS (Agent-Native Multimodal GEneration with Memory and Skills), a framework that pushes beyond the inherent limitations of foundational models on both general and downstream tasks. GEMS is built upon three core components. Agent Loop introduces a structured multi-agent framework that iteratively improves generation quality through closed-loop optimization. Agent Memory provides a persistent, trajectory-level memory that hierarchically stores both factual states and compressed experiential summaries, enabling a global view of the optimization process while reducing redundancy. Agent Skill offers an extensible collection of domain-specific expertise with on-demand loading, allowing the system to effectively handle diverse downstream applications. Across five mainstream tasks and four downstream tasks, evaluated on multiple generative backends, GEMS consistently achieves significant performance gains. Most notably, it enables the lightweight 6B model Z-Image-Turbo to surpass the state-of-the-art Nano Banana 2 on GenEval2, demonstrating the effectiveness of agent harness in extending model capabilities beyond their original limits.
69cb6efe22f9e9dce3fe21c4
Zefeng He
claimed_verified
2026-03-31T07:41:03.099Z
67247adb73d1eb17b6bfd27c
/avatars/57bdbb7362f9854c87dd0a71ae071652.svg
Zefeng He
yhx12
user
yhx12
69cb6efe22f9e9dce3fe21c5
Siyuan Huang
claimed_verified
2026-04-01T08:36:11.268Z
66f79b323fe089b75e9e0c04
/avatars/92918bf8913012a3f005f09e03b381c2.svg
Siyuan Huang
chamber111
user
chamber111
69cb6efe22f9e9dce3fe21c6
Xiaoye Qu
69cb6efe22f9e9dce3fe21c7
Yafu Li
69cb6efe22f9e9dce3fe21c8
Tong Zhu
69cb6efe22f9e9dce3fe21c9
Yu Cheng
69cb6efe22f9e9dce3fe21ca
Yang Yang
GEMS: Agent-Native Multimodal Generation with Memory and Skills
Recent multimodal generation models have achieved remarkable progress on general-purpose generation tasks, yet continue to struggle with complex instructions and specialized downstream tasks. Inspired by the success of advanced agent frameworks such as Claude Code, we propose GEMS (Agent-Native Multimodal GEneration with Memory and Skills), a framework that pushes beyond the inherent limitations of foundational models on both general and downstream tasks. GEMS is built upon three core components. Agent Loop introduces a structured multi-agent framework that iteratively improves generation quality through closed-loop optimization. Agent Memory provides a persistent, trajectory-level memory that hierarchically stores both factual states and compressed experiential summaries, enabling a global view of the optimization process while reducing redundancy. Agent Skill offers an extensible collection of domain-specific expertise with on-demand loading, allowing the system to effectively handle diverse downstream applications. Across five mainstream tasks and four downstream tasks, evaluated on multiple generative backends, GEMS consistently achieves significant performance gains. Most notably, it enables the lightweight 6B model Z-Image-Turbo to surpass the state-of-the-art Nano Banana 2 on GenEval2, demonstrating the effectiveness of agent harness in extending model capabilities beyond their original limits.
GEMS is an agent-native multimodal generation framework that enhances model capabilities through structured multi-agent optimization, persistent memory, and domain-specific skills across general and downstream tasks.
multimodal generation models agent frameworks agent loop agent memory agent skill closed-loop optimization trajectory-level memory domain-specific expertise generative backends Z-Image-Turbo Nano Banana 2