VTCBench: Can Vision-Language Models Understand Long Context with Vision-Text Compression?
The computational and memory overheads associated with expanding the context window of LLMs severely limit their scalability. A noteworthy solution is vision-text compression (VTC), exemplified by frameworks like DeepSeek-OCR and Glyph, which convert long texts into dense 2D visual representations, thereby achieving token compression ratios of 3x-20x. However, the impact of this high information density on the core long-context capabilities of vision-language models (VLMs) remains under-investigated. To address this gap, we introduce the first benchmark for VTC and systematically assess the performance of VLMs across three long-context understanding settings: VTC-Retrieval, which evaluates the model's ability to retrieve and aggregate information; VTC-Reasoning, which requires models to infer latent associations to locate facts with minimal lexical overlap; and VTC-Memory, which measures comprehensive question answering within long-term dialogue memory. Furthermore, we establish the VTCBench-Wild to simulate diverse input scenarios.We comprehensively evaluate leading open-source and proprietary models on our benchmarks. The results indicate that, despite being able to decode textual information (e.g., OCR) well, most VLMs exhibit a surprisingly poor long-context understanding ability with VTC-compressed information, failing to capture long associations or dependencies in the context.This study provides a deep understanding of VTC and serves as a foundation for designing more efficient and scalable VLMs.
69436833542d62d58a7bf648
Hongbo Zhao
claimed_verified
2025-12-18T08:00:05.012Z
64100834c025ddf6189c415e
/avatars/9b9bbecef5d5815540abf92d74012f55.svg
Hongbo Zhao
z-hb
user
z-hb
69436833542d62d58a7bf649
Meng Wang
claimed_verified
2025-12-18T08:00:03.214Z
655698848d9e07913d6c8620
https://cdn-avatars.huggingface.co/v1/production/uploads/655698848d9e07913d6c8620/UeLV4MGfOmpEHJHKFms2r.jpeg
Meng Wang
Moenupa
user
Moenupa
69436833542d62d58a7bf64a
Fei Zhu
69436833542d62d58a7bf64b
Wenzhuo Liu
69436833542d62d58a7bf64c
Bolin Ni
69436833542d62d58a7bf64d
Fanhu Zeng
claimed_verified
2025-12-22T11:00:52.114Z
667b7388580a979b703c3c2b
/avatars/15d42bf75f88f1e2ac112c98cb53bad6.svg
Fanhu Zeng
AuroraZengfh
user
AuroraZengfh
69436833542d62d58a7bf64e
Gaofeng Meng
69436833542d62d58a7bf64f
Zhaoxiang Zhang
VTCBench: Can Vision-Language Models Understand Long Context with Vision-Text Compression?
The computational and memory overheads associated with expanding the context window of LLMs severely limit their scalability. A noteworthy solution is vision-text compression (VTC), exemplified by frameworks like DeepSeek-OCR and Glyph, which convert long texts into dense 2D visual representations, thereby achieving token compression ratios of 3x-20x. However, the impact of this high information density on the core long-context capabilities of vision-language models (VLMs) remains under-investigated. To address this gap, we introduce the first benchmark for VTC and systematically assess the performance of VLMs across three long-context understanding settings: VTC-Retrieval, which evaluates the model's ability to retrieve and aggregate information; VTC-Reasoning, which requires models to infer latent associations to locate facts with minimal lexical overlap; and VTC-Memory, which measures comprehensive question answering within long-term dialogue memory. Furthermore, we establish the VTCBench-Wild to simulate diverse input scenarios.We comprehensively evaluate leading open-source and proprietary models on our benchmarks. The results indicate that, despite being able to decode textual information (e.g., OCR) well, most VLMs exhibit a surprisingly poor long-context understanding ability with VTC-compressed information, failing to capture long associations or dependencies in the context.This study provides a deep understanding of VTC and serves as a foundation for designing more efficient and scalable VLMs.
Vision-text compression frameworks convert long texts into visual representations to reduce context window overhead, but current vision-language models struggle with long-context understanding when processing such compressed information.
vision-text compression vision-language models context window token compression VTC-Retrieval VTC-Reasoning VTC-Memory VTCBench-Wild OCR