DLEBench: Evaluating Small-scale Object Editing Ability for Instruction-based Image Editing Model
Significant progress has been made in the field of Instruction-based Image Editing Models (IIEMs). However, while these models demonstrate plausible adherence to instructions and strong reasoning ability on current benchmarks, their ability to edit small objects remains underexplored, despite its importance for precise local editing and refining details in both real and generated images. In this paper, we introduce DeepLookEditBench (DLEBench), the first benchmark dedicated to assessing the abilities of IIEMs in editing small-scale objects. Specifically, we construct a challenging testbed comprising 1889 samples across seven instruction types. In these samples, target objects occupy only 1%-10% of the image area, covering complex scenarios such as partial occlusion and multi-object editing. To ensure robust evaluation on this benchmark, we propose an evaluation protocol with refined score rubrics to minimize subjectivity and ambiguity in two criteria: Instruction Following and Visual Consistency. This protocol also introduces a dual-mode evaluation framework (Tool-driven and Oracle-guided Modes) addressing the misalignment between LMM-as-a-Judge and human judgements on DLEBench. Empirical results on 10 IIEMs reveal significant performance gaps in small-scale object editing, highlighting the need for specialized benchmarks to advance this ability.
69a4f19cbc2d66e68888f496
Shibo Hong
69a4f19cbc2d66e68888f497
Boxian Ai
claimed_verified
2026-03-09T10:39:21.906Z
68e9d5540a03eedd74881313
https://cdn-avatars.huggingface.co/v1/production/uploads/68e9d5540a03eedd74881313/NmiP7yZsF0DA9G35q1zEW.jpeg
Boxian Ai
Loasaster
user
Loasaster
69a4f19cbc2d66e68888f498
Jun Kuang
claimed_verified
2026-04-15T12:09:47.987Z
655c7bb799c64cb090c76cf8
/avatars/060df85cfe4184907084e6d909ea6a1c.svg
Jun Kuang
JKuang96
user
JKuang96
69a4f19cbc2d66e68888f499
Wei Wang
69a4f19cbc2d66e68888f49a
FengJiao Chen
69a4f19cbc2d66e68888f49b
Zhongyuan Peng
69a4f19cbc2d66e68888f49c
Chenhao Huang
69a4f19cbc2d66e68888f49d
Yixin Cao
DLEBench: Evaluating Small-scale Object Editing Ability for Instruction-based Image Editing Model
Significant progress has been made in the field of Instruction-based Image Editing Models (IIEMs). However, while these models demonstrate plausible adherence to instructions and strong reasoning ability on current benchmarks, their ability to edit small objects remains underexplored, despite its importance for precise local editing and refining details in both real and generated images. In this paper, we introduce DeepLookEditBench (DLEBench), the first benchmark dedicated to assessing the abilities of IIEMs in editing small-scale objects. Specifically, we construct a challenging testbed comprising 1889 samples across seven instruction types. In these samples, target objects occupy only 1%-10% of the image area, covering complex scenarios such as partial occlusion and multi-object editing. To ensure robust evaluation on this benchmark, we propose an evaluation protocol with refined score rubrics to minimize subjectivity and ambiguity in two criteria: Instruction Following and Visual Consistency. This protocol also introduces a dual-mode evaluation framework (Tool-driven and Oracle-guided Modes) addressing the misalignment between LMM-as-a-Judge and human judgements on DLEBench. Empirical results on 10 IIEMs reveal significant performance gaps in small-scale object editing, highlighting the need for specialized benchmarks to advance this ability.
A new benchmark called DeepLookEditBench is introduced to evaluate instruction-based image editing models' capability in handling small-scale object editing, revealing significant performance gaps in this area.
Instruction-based Image Editing Models small-scale object editing benchmark evaluation protocol score rubrics dual-mode evaluation framework Tool-driven mode Oracle-guided mode