UniREditBench: A Unified Reasoning-based Image Editing Benchmark
Recent advances in multi-modal generative models have driven substantial
improvements in image editing. However, current generative models still
struggle with handling diverse and complex image editing tasks that require
implicit reasoning, underscoring the need for a comprehensive benchmark to
systematically assess their performance across various reasoning scenarios.
Existing benchmarks primarily focus on single-object attribute transformation
in realistic scenarios, which, while effective, encounter two key challenges:
(1) they largely overlook multi-object interactions as well as game-world
scenarios that involve human-defined rules, which are common in real-life
applications; (2) they only rely on textual references to evaluate the
generated images, potentially leading to systematic misjudgments, especially in
complex reasoning scenarios. To this end, this work proposes UniREditBench, a
unified benchmark for reasoning-based image editing evaluation. It comprises
2,700 meticulously curated samples, covering both real- and game-world
scenarios across 8 primary dimensions and 18 sub-dimensions. To improve
evaluation reliability, we introduce multimodal dual-reference evaluation,
providing both textual and ground-truth image references for each sample
assessment. Furthermore, we design an automated multi-scenario data synthesis
pipeline and construct UniREdit-Data-100K, a large-scale synthetic dataset with
high-quality chain-of-thought (CoT) reasoning annotations. We fine-tune Bagel
on this dataset and develop UniREdit-Bagel, demonstrating substantial
improvements in both in-domain and out-of-distribution settings. Through
thorough benchmarking of both open-source and closed-source image editing
models, we reveal their strengths and weaknesses across various aspects.
69097a04812eca10f9cc62fe
Feng Han
69097a04812eca10f9cc62ff
Yibin Wang
claimed_verified
2025-11-04T20:24:43.308Z
654c6845bac6e6e49895a5b5
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/KXQaAxulqr8jNBSpEaYM4.png
SII-Yibin Wang
CodeGoat24
user
CodeGoat24
69097a04812eca10f9cc6300
Chenglin Li
69097a04812eca10f9cc6301
Zheming Liang
69097a04812eca10f9cc6302
Dianyi Wang
claimed_verified
2026-02-02T17:03:19.419Z
66084749d81d611249a18239
/avatars/a7256b4e3b686133aaa481a72e3058de.svg
Alex Wang(SII)
Alex11556666
user
Alex11556666
69097a04812eca10f9cc6303
Yang Jiao
69097a04812eca10f9cc6304
Zhipeng Wei
69097a04812eca10f9cc6305
Chao Gong
69097a04812eca10f9cc6306
Cheng Jin
69097a04812eca10f9cc6307
Jingjing Chen
69097a04812eca10f9cc6308
Jiaqi Wang
claimed_verified
2026-02-17T15:48:30.883Z
64b4eec4faa3181a5eab9c46
/avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
Jiaqi Wang
myownskyW7
user
myownskyW7
UniREditBench: A Unified Reasoning-based Image Editing Benchmark
Recent advances in multi-modal generative models have driven substantial
improvements in image editing. However, current generative models still
struggle with handling diverse and complex image editing tasks that require
implicit reasoning, underscoring the need for a comprehensive benchmark to
systematically assess their performance across various reasoning scenarios.
Existing benchmarks primarily focus on single-object attribute transformation
in realistic scenarios, which, while effective, encounter two key challenges:
(1) they largely overlook multi-object interactions as well as game-world
scenarios that involve human-defined rules, which are common in real-life
applications; (2) they only rely on textual references to evaluate the
generated images, potentially leading to systematic misjudgments, especially in
complex reasoning scenarios. To this end, this work proposes UniREditBench, a
unified benchmark for reasoning-based image editing evaluation. It comprises
2,700 meticulously curated samples, covering both real- and game-world
scenarios across 8 primary dimensions and 18 sub-dimensions. To improve
evaluation reliability, we introduce multimodal dual-reference evaluation,
providing both textual and ground-truth image references for each sample
assessment. Furthermore, we design an automated multi-scenario data synthesis
pipeline and construct UniREdit-Data-100K, a large-scale synthetic dataset with
high-quality chain-of-thought (CoT) reasoning annotations. We fine-tune Bagel
on this dataset and develop UniREdit-Bagel, demonstrating substantial
improvements in both in-domain and out-of-distribution settings. Through
thorough benchmarking of both open-source and closed-source image editing
models, we reveal their strengths and weaknesses across various aspects.
UniREditBench is a unified benchmark for reasoning-based image editing that addresses limitations in existing benchmarks by including multi-object interactions, game-world scenarios, and multimodal dual-reference evaluation.
multi-modal generative models image editing implicit reasoning benchmark single-object attribute transformation real-world scenarios game-world scenarios human-defined rules multimodal dual-reference evaluation automated multi-scenario data synthesis chain-of-thought reasoning fine-tuning UniREdit-Data-100K UniREdit-Bagel in-domain out-of-distribution