MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generation
Recent text-to-image generation models have acquired the ability of multi-reference generation and editing; the ability to inherit the appearance of subjects from multiple reference images and re-render them under new contexts. However, the existing benchmark datasets often focus on the generation with single or a few reference images, which prevents us from measuring the progress on how model performance advances or pointing out their weaknesses, under different multi-reference conditions. In addition, their task definitions are still vague, typically limited to axes such as "what to edit" or "how many references are given", and therefore fail to capture the intrinsic difficulty of multi-reference settings. To address this gap, we introduce MultiBanana, which is carefully designed to assesses the edge of model capabilities by widely covering multi-reference-specific problems at scale: (1) varying the number of references, (2) domain mismatch among references (e.g., photo vs. anime), (3) scale mismatch between reference and target scenes, (4) references containing rare concepts (e.g., a red banana), and (5) multilingual textual references for rendering. Our analysis among a variety of text-to-image models reveals their superior performances, typical failure modes, and areas for improvement. MultiBanana will be released as an open benchmark to push the boundaries and establish a standardized basis for fair comparison in multi-reference image generation. Our data and code are available at https://github.com/matsuolab/multibanana .
692d229d4397b1ec214f6902
Yuta Oshima
claimed_verified
2025-12-01T09:10:18.876Z
666a940b0abf7b9054e583c9
/avatars/ca2399de0eba0b764b99aa8539275849.svg
Yuta Oshima
shim0114
user
shim0114
692d229d4397b1ec214f6903
Daiki Miyake
claimed_verified
2025-12-03T09:18:48.374Z
628be863e410cb05ea0590e0
/avatars/9be9cd023ebdffe52ac0b6a2e8ec228a.svg
Miyake Daiki
KYM384
user
KYM384
692d229d4397b1ec214f6904
Kohsei Matsutani
claimed_verified
2025-12-01T09:10:20.587Z
64618cf05dba83471db2be9b
/avatars/0170d53d2df272903c09beb5336eeee5.svg
Kohsei Matsutani
kohsei
user
kohsei
692d229d4397b1ec214f6905
Yusuke Iwasawa
692d229d4397b1ec214f6906
Masahiro Suzuki
692d229d4397b1ec214f6907
Yutaka Matsuo
692d229d4397b1ec214f6908
Hiroki Furuta
claimed_verified
2025-12-04T08:49:42.136Z
646c39fc3e2a7b06594bac6c
/avatars/8925fef2b6e7105a0f0a4bbf160cfda3.svg
Hiroki Furuta
frt03
user
frt03
MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generation
Recent text-to-image generation models have acquired the ability of multi-reference generation and editing; the ability to inherit the appearance of subjects from multiple reference images and re-render them under new contexts. However, the existing benchmark datasets often focus on the generation with single or a few reference images, which prevents us from measuring the progress on how model performance advances or pointing out their weaknesses, under different multi-reference conditions. In addition, their task definitions are still vague, typically limited to axes such as "what to edit" or "how many references are given", and therefore fail to capture the intrinsic difficulty of multi-reference settings. To address this gap, we introduce MultiBanana, which is carefully designed to assesses the edge of model capabilities by widely covering multi-reference-specific problems at scale: (1) varying the number of references, (2) domain mismatch among references (e.g., photo vs. anime), (3) scale mismatch between reference and target scenes, (4) references containing rare concepts (e.g., a red banana), and (5) multilingual textual references for rendering. Our analysis among a variety of text-to-image models reveals their superior performances, typical failure modes, and areas for improvement. MultiBanana will be released as an open benchmark to push the boundaries and establish a standardized basis for fair comparison in multi-reference image generation. Our data and code are available at https://github.com/matsuolab/multibanana .
MultiBanana is a benchmark dataset for evaluating multi-reference text-to-image generation models across various challenging conditions, providing insights into model strengths and weaknesses.
text-to-image generation multi-reference generation multi-reference editing benchmark datasets domain mismatch scale mismatch rare concepts multilingual textual references