Open Multimodal Retrieval-Augmented Factual Image Generation
Large Multimodal Models (LMMs) have achieved remarkable progress in
generating photorealistic and prompt-aligned images, but they often produce
outputs that contradict verifiable knowledge, especially when prompts involve
fine-grained attributes or time-sensitive events. Conventional
retrieval-augmented approaches attempt to address this issue by introducing
external information, yet they are fundamentally incapable of grounding
generation in accurate and evolving knowledge due to their reliance on static
sources and shallow evidence integration. To bridge this gap, we introduce
ORIG, an agentic open multimodal retrieval-augmented framework for Factual
Image Generation (FIG), a new task that requires both visual realism and
factual grounding. ORIG iteratively retrieves and filters multimodal evidence
from the web and incrementally integrates the refined knowledge into enriched
prompts to guide generation. To support systematic evaluation, we build
FIG-Eval, a benchmark spanning ten categories across perceptual, compositional,
and temporal dimensions. Experiments demonstrate that ORIG substantially
improves factual consistency and overall image quality over strong baselines,
highlighting the potential of open multimodal retrieval for factual image
generation.
6900782d646208eac0d1ef17
Yang Tian
claimed_verified
2025-10-28T15:26:45.865Z
6565d47e75968d50cc2735ac
https://cdn-avatars.huggingface.co/v1/production/uploads/6565d47e75968d50cc2735ac/eX-h3GWaudhJii1IS7v8K.jpeg
Yang Tian
TyangJN
user
TyangJN
6900782d646208eac0d1ef18
Fan Liu
6900782d646208eac0d1ef19
Jingyuan Zhang
claimed_verified
2025-11-06T13:45:50.784Z
64b20f6b102ed6e7aece9d13
/avatars/194b11ce0e3206cc7a706201af721281.svg
jingyuan zhang
crazy-dahuzi
user
crazy-dahuzi
6900782d646208eac0d1ef1a
Wei Bi
6900782d646208eac0d1ef1b
Yupeng Hu
6900782d646208eac0d1ef1c
Liqiang Nie
Open Multimodal Retrieval-Augmented Factual Image Generation
Large Multimodal Models (LMMs) have achieved remarkable progress in
generating photorealistic and prompt-aligned images, but they often produce
outputs that contradict verifiable knowledge, especially when prompts involve
fine-grained attributes or time-sensitive events. Conventional
retrieval-augmented approaches attempt to address this issue by introducing
external information, yet they are fundamentally incapable of grounding
generation in accurate and evolving knowledge due to their reliance on static
sources and shallow evidence integration. To bridge this gap, we introduce
ORIG, an agentic open multimodal retrieval-augmented framework for Factual
Image Generation (FIG), a new task that requires both visual realism and
factual grounding. ORIG iteratively retrieves and filters multimodal evidence
from the web and incrementally integrates the refined knowledge into enriched
prompts to guide generation. To support systematic evaluation, we build
FIG-Eval, a benchmark spanning ten categories across perceptual, compositional,
and temporal dimensions. Experiments demonstrate that ORIG substantially
improves factual consistency and overall image quality over strong baselines,
highlighting the potential of open multimodal retrieval for factual image
generation.
ORIG, an agentic open multimodal retrieval-augmented framework, enhances factual consistency and image quality in factual image generation by iteratively integrating refined web-based evidence into prompts.
Large Multimodal Models factual image generation retrieval-augmented multimodal evidence perceptual compositional temporal dimensions factual consistency