Enhancing Multi-Image Understanding through Delimiter Token Scaling
Large Vision-Language Models (LVLMs) achieve strong performance on single-image tasks, but their performance declines when multiple images are provided as input. One major reason is the cross-image information leakage, where the model struggles to distinguish information across different images. Existing LVLMs already employ delimiter tokens to mark the start and end of each image, yet our analysis reveals that these tokens fail to effectively block cross-image information leakage. To enhance their effectiveness, we propose a method that scales the hidden states of delimiter tokens. This enhances the model's ability to preserve image-specific information by reinforcing intra-image interaction and limiting undesired cross-image interactions. Consequently, the model is better able to distinguish between images and reason over them more accurately. Experiments show performance gains on multi-image benchmarks such as Mantis, MuirBench, MIRB, and QBench2. We further evaluate our method on text-only tasks that require clear distinction. The method improves performance on multi-document and multi-table understanding benchmarks, including TQABench, MultiNews, and WCEP-10. Notably, our method requires no additional training or inference cost.
698182d1ce18b186280962f4
Minyoung Lee
claimed_verified
2026-02-03T10:03:57.079Z
676ac5647fff9075b50fb214
https://cdn-avatars.huggingface.co/v1/production/uploads/676ac5647fff9075b50fb214/n0cODUCzZDPHa6iZQ3XDR.jpeg
Minyoung
MYMY-young
user
MYMY-young
698182d1ce18b186280962f5
Yeji Park
698182d1ce18b186280962f6
Dongjun Hwang
698182d1ce18b186280962f7
Yejin Kim
698182d1ce18b186280962f8
Seong Joon Oh
698182d1ce18b186280962f9
Junsuk Choe
Enhancing Multi-Image Understanding through Delimiter Token Scaling
Large Vision-Language Models (LVLMs) achieve strong performance on single-image tasks, but their performance declines when multiple images are provided as input. One major reason is the cross-image information leakage, where the model struggles to distinguish information across different images. Existing LVLMs already employ delimiter tokens to mark the start and end of each image, yet our analysis reveals that these tokens fail to effectively block cross-image information leakage. To enhance their effectiveness, we propose a method that scales the hidden states of delimiter tokens. This enhances the model's ability to preserve image-specific information by reinforcing intra-image interaction and limiting undesired cross-image interactions. Consequently, the model is better able to distinguish between images and reason over them more accurately. Experiments show performance gains on multi-image benchmarks such as Mantis, MuirBench, MIRB, and QBench2. We further evaluate our method on text-only tasks that require clear distinction. The method improves performance on multi-document and multi-table understanding benchmarks, including TQABench, MultiNews, and WCEP-10. Notably, our method requires no additional training or inference cost.
Scaling hidden states of delimiter tokens in vision-language models reduces cross-image information leakage and improves multi-image reasoning performance.
vision-language models cross-image information leakage delimiter tokens hidden states intra-image interaction cross-image interactions multi-image benchmarks Mantis MuirBench MIRB QBench2 multi-document understanding multi-table understanding TQABench MultiNews WCEP-10