Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis
Unified multimodal models provide a natural and promising architecture for understanding diverse and complex real-world knowledge while generating high-quality images. However, they still rely primarily on frozen parametric knowledge, which makes them struggle with real-world image generation involving long-tail and knowledge-intensive concepts. Inspired by the broad success of agents on real-world tasks, we explore agentic modeling to address this limitation. Specifically, we present Unify-Agent, a unified multimodal agent for world-grounded image synthesis, which reframes image generation as an agentic pipeline consisting of prompt understanding, multimodal evidence searching, grounded recaptioning, and final synthesis. To train our model, we construct a tailored multimodal data pipeline and curate 143K high-quality agent trajectories for world-grounded image synthesis, enabling effective supervision over the full agentic generation process. We further introduce FactIP, a benchmark covering 12 categories of culturally significant and long-tail factual concepts that explicitly requires external knowledge grounding. Extensive experiments show that our proposed Unify-Agent substantially improves over its base unified model across diverse benchmarks and real world generation tasks, while approaching the world knowledge capabilities of the strongest closed-source models. As an early exploration of agent-based modeling for world-grounded image synthesis, our work highlights the value of tightly coupling reasoning, searching, and generation for reliable open-world agentic image synthesis.
69cc8d043f84a0971be66b9e
Shuang Chen
claimed_verified
2026-04-07T08:46:59.765Z
66cc19ad6f8945277c39cd86
/avatars/f031e77d7fb9dbebbc00fba9b5dd5357.svg
Shawn
csfufu
user
csfufu
69cc8d043f84a0971be66b9f
Quanxin Shou
69cc8d043f84a0971be66ba0
Hangting Chen
69cc8d043f84a0971be66ba1
Yucheng Zhou
claimed_verified
2026-04-01T08:34:31.506Z
636f37fa93d9a0c987e092fa
https://cdn-avatars.huggingface.co/v1/production/uploads/636f37fa93d9a0c987e092fa/vdZgFPobSIUbBTC3jlfH5.jpeg
Yucheng Zhou
YCZhou
user
YCZhou
69cc8d043f84a0971be66ba2
Kaituo Feng
69cc8d043f84a0971be66ba3
Wenbo Hu
69cc8d043f84a0971be66ba4
Yi-Fan Zhang
claimed_verified
2026-04-08T08:57:26.110Z
623d8ca4c29adf5ef6175615
https://cdn-avatars.huggingface.co/v1/production/uploads/623d8ca4c29adf5ef6175615/q7lHao7UPwU1u7YLSP56m.jpeg
Yi-Fan Zhang
yifanzhang114
user
yifanzhang114
69cc8d043f84a0971be66ba5
Yunlong Lin
claimed_verified
2026-04-01T08:34:33.580Z
64ecb174f22081b4ac7ca397
https://cdn-avatars.huggingface.co/v1/production/uploads/64ecb174f22081b4ac7ca397/PiAPtD_rbuhGOqfE6ZSIu.jpeg
Yunlong Lin
LYL1015
user
LYL1015
69cc8d043f84a0971be66ba6
Wenxuan Huang
69cc8d043f84a0971be66ba7
Mingyang Song
69cc8d043f84a0971be66ba8
Dasen Dai
69cc8d043f84a0971be66ba9
Bolin Jiang
69cc8d043f84a0971be66baa
Manyuan Zhang
69cc8d043f84a0971be66bab
Shi-Xue Zhang
69cc8d043f84a0971be66bac
Zhengkai Jiang
69cc8d043f84a0971be66bad
Lucas Wang
69cc8d043f84a0971be66bae
Zhao Zhong
69cc8d043f84a0971be66baf
Yu Cheng
69cc8d043f84a0971be66bb0
Nanyun Peng
Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis
Unified multimodal models provide a natural and promising architecture for understanding diverse and complex real-world knowledge while generating high-quality images. However, they still rely primarily on frozen parametric knowledge, which makes them struggle with real-world image generation involving long-tail and knowledge-intensive concepts. Inspired by the broad success of agents on real-world tasks, we explore agentic modeling to address this limitation. Specifically, we present Unify-Agent, a unified multimodal agent for world-grounded image synthesis, which reframes image generation as an agentic pipeline consisting of prompt understanding, multimodal evidence searching, grounded recaptioning, and final synthesis. To train our model, we construct a tailored multimodal data pipeline and curate 143K high-quality agent trajectories for world-grounded image synthesis, enabling effective supervision over the full agentic generation process. We further introduce FactIP, a benchmark covering 12 categories of culturally significant and long-tail factual concepts that explicitly requires external knowledge grounding. Extensive experiments show that our proposed Unify-Agent substantially improves over its base unified model across diverse benchmarks and real world generation tasks, while approaching the world knowledge capabilities of the strongest closed-source models. As an early exploration of agent-based modeling for world-grounded image synthesis, our work highlights the value of tightly coupling reasoning, searching, and generation for reliable open-world agentic image synthesis.
Unify-Agent integrates agent-based modeling with multimodal understanding to enhance image synthesis through reasoning, searching, and generation processes grounded in external knowledge.
unified multimodal models agentic modeling world-grounded image synthesis prompt understanding multimodal evidence searching grounded recaptioning agent trajectories FactIP closed-source models