CodeOCR: On the Effectiveness of Vision Language Models in Code Understanding
Large Language Models (LLMs) have achieved remarkable success in source code understanding, yet as software systems grow in scale, computational efficiency has become a critical bottleneck. Currently, these models rely on a text-based paradigm that treats source code as a linear sequence of tokens, which leads to a linear increase in context length and associated computational costs. The rapid advancement of Multimodal LLMs (MLLMs) introduces an opportunity to optimize efficiency by representing source code as rendered images. Unlike text, which is difficult to compress without losing semantic meaning, the image modality is inherently suitable for compression. By adjusting resolution, images can be scaled to a fraction of their original token cost while remaining recognizable to vision-capable models. To explore the feasibility of this approach, we conduct the first systematic study on the effectiveness of MLLMs for code understanding. Our experiments reveal that: (1) MLLMs can effectively understand code with substantial token reduction, achieving up to 8x compression; (2) MLLMs can effectively leverage visual cues such as syntax highlighting, improving code completion performance under 4x compression; and (3) Code-understanding tasks like clone detection exhibit exceptional resilience to visual compression, with some compression ratios even slightly outperforming raw text inputs. Our findings highlight both the potential and current limitations of MLLMs in code understanding, which points out a shift toward image-modality code representation as a pathway to more efficient inference.
69818a88ce18b18628096389
Yuling Shi
claimed_verified
2026-02-03T10:03:06.748Z
645b0c3ec35da9c7afd95421
https://cdn-avatars.huggingface.co/v1/production/uploads/645b0c3ec35da9c7afd95421/vYBrCDagHsXAo6J2p-uG0.jpeg
Yuling
YerbaPage
user
YerbaPage
69818a88ce18b1862809638a
Chaoxiang Xie
claimed_verified
2026-02-04T12:31:45.296Z
68354b3e65397cd063da14e4
/avatars/1b85fc942b41f58dac8e72bd12dd8e55.svg
Chaoxiang Xie
bailynlove
user
bailynlove
69818a88ce18b1862809638b
Zhensu Sun
claimed_verified
2026-02-23T09:47:32.835Z
6295c00307dbe3912697e983
https://cdn-avatars.huggingface.co/v1/production/uploads/6295c00307dbe3912697e983/J6Eyes3Xurv0bvjTbjTjT.jpeg
v587su
zhensuuu
user
zhensuuu
69818a88ce18b1862809638c
Yeheng Chen
69818a88ce18b1862809638d
Chenxu Zhang
claimed_verified
2026-02-05T10:56:32.330Z
6981bddf921842423e121142
/avatars/7435a8bebda29bfdfa6d4b4c7bfe28d5.svg
Chenxu Zhang
Tassels-zcx
user
Tassels-zcx
69818a88ce18b1862809638e
Longfei Yun
69818a88ce18b1862809638f
Chengcheng Wan
69818a88ce18b18628096390
Hongyu Zhang
69818a88ce18b18628096391
David Lo
69818a88ce18b18628096392
Xiaodong Gu
CodeOCR: On the Effectiveness of Vision Language Models in Code Understanding
Large Language Models (LLMs) have achieved remarkable success in source code understanding, yet as software systems grow in scale, computational efficiency has become a critical bottleneck. Currently, these models rely on a text-based paradigm that treats source code as a linear sequence of tokens, which leads to a linear increase in context length and associated computational costs. The rapid advancement of Multimodal LLMs (MLLMs) introduces an opportunity to optimize efficiency by representing source code as rendered images. Unlike text, which is difficult to compress without losing semantic meaning, the image modality is inherently suitable for compression. By adjusting resolution, images can be scaled to a fraction of their original token cost while remaining recognizable to vision-capable models. To explore the feasibility of this approach, we conduct the first systematic study on the effectiveness of MLLMs for code understanding. Our experiments reveal that: (1) MLLMs can effectively understand code with substantial token reduction, achieving up to 8x compression; (2) MLLMs can effectively leverage visual cues such as syntax highlighting, improving code completion performance under 4x compression; and (3) Code-understanding tasks like clone detection exhibit exceptional resilience to visual compression, with some compression ratios even slightly outperforming raw text inputs. Our findings highlight both the potential and current limitations of MLLMs in code understanding, which points out a shift toward image-modality code representation as a pathway to more efficient inference.
Multimodal large language models can effectively understand source code when represented as compressed images, achieving significant token reduction while maintaining or improving performance on code comprehension tasks.
Multimodal LLMs source code understanding token compression visual cues syntax highlighting code completion clone detection image modality visual compression computational efficiency