IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Prompting
Recent advances in multimodal large language models (MLLMs) have led to impressive progress across various benchmarks. However, their capability in understanding infrared images remains unexplored. To address this gap, we introduce IF-Bench, the first high-quality benchmark designed for evaluating multimodal understanding of infrared images. IF-Bench consists of 499 images sourced from 23 infrared datasets and 680 carefully curated visual question-answer pairs, covering 10 essential dimensions of image understanding. Based on this benchmark, we systematically evaluate over 40 open-source and closed-source MLLMs, employing cyclic evaluation, bilingual assessment, and hybrid judgment strategies to enhance the reliability of the results. Our analysis reveals how model scale, architecture, and inference paradigms affect infrared image comprehension, providing valuable insights for this area. Furthermore, we propose a training-free generative visual prompting (GenViP) method, which leverages advanced image editing models to translate infrared images into semantically and spatially aligned RGB counterparts, thereby mitigating domain distribution shifts. Extensive experiments demonstrate that our method consistently yields significant performance improvements across a wide range of MLLMs. The benchmark and code are available at https://github.com/casiatao/IF-Bench.
693a4a8b74fced5bf9c32489
Tao Zhang
claimed_verified
2025-12-12T09:16:59.360Z
66ceeab7528c51fee3d2a25e
/avatars/23833573f668abc1bf343cca79b3d7a0.svg
zhangtao
casiatao
user
casiatao
693a4a8b74fced5bf9c3248a
Yuyang Hong
claimed_verified
2025-12-11T10:12:37.559Z
66e03d7123e5f162e7e5c5ac
https://cdn-avatars.huggingface.co/v1/production/uploads/66e03d7123e5f162e7e5c5ac/PHHGlQKnZhr_BNpciB9Zz.jpeg
hongyuyang
hongyuyang23casia
user
hongyuyang23casia
693a4a8b74fced5bf9c3248b
Yang Xia
693a4a8b74fced5bf9c3248c
Kun Ding
693a4a8b74fced5bf9c3248d
Zeyu Zhang
693a4a8b74fced5bf9c3248e
Ying Wang
693a4a8b74fced5bf9c3248f
Shiming Xiang
693a4a8b74fced5bf9c32490
Chunhong Pan
IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Prompting
Recent advances in multimodal large language models (MLLMs) have led to impressive progress across various benchmarks. However, their capability in understanding infrared images remains unexplored. To address this gap, we introduce IF-Bench, the first high-quality benchmark designed for evaluating multimodal understanding of infrared images. IF-Bench consists of 499 images sourced from 23 infrared datasets and 680 carefully curated visual question-answer pairs, covering 10 essential dimensions of image understanding. Based on this benchmark, we systematically evaluate over 40 open-source and closed-source MLLMs, employing cyclic evaluation, bilingual assessment, and hybrid judgment strategies to enhance the reliability of the results. Our analysis reveals how model scale, architecture, and inference paradigms affect infrared image comprehension, providing valuable insights for this area. Furthermore, we propose a training-free generative visual prompting (GenViP) method, which leverages advanced image editing models to translate infrared images into semantically and spatially aligned RGB counterparts, thereby mitigating domain distribution shifts. Extensive experiments demonstrate that our method consistently yields significant performance improvements across a wide range of MLLMs. The benchmark and code are available at https://github.com/casiatao/IF-Bench.
The introduction of IF-Bench evaluates multimodal large language models' performance on infrared images using diverse assessment strategies and proposes a training-free method to improve comprehension.
multimodal large language models MLLMs IF-Bench infrared images visual question-answer pairs cyclic evaluation bilingual assessment hybrid judgment strategies model scale architecture inference paradigms generative visual prompting GenViP domain distribution shifts