LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs
Transforming a large language model (LLM) into a Vision-Language Model (VLM) can be achieved by mapping the visual tokens from a vision encoder into the embedding space of an LLM. Intriguingly, this mapping can be as simple as a shallow MLP transformation. To understand why LLMs can so readily process visual tokens, we need interpretability methods that reveal what is encoded in the visual token representations at every layer of LLM processing. In this work, we introduce LatentLens, a novel approach for mapping latent representations to descriptions in natural language. LatentLens works by encoding a large text corpus and storing contextualized token representations for each token in that corpus. Visual token representations are then compared to their contextualized textual representations, with the top-k nearest neighbor representations providing descriptions of the visual token. We evaluate this method on 10 different VLMs, showing that commonly used methods, such as LogitLens, substantially underestimate the interpretability of visual tokens. With LatentLens instead, the majority of visual tokens are interpretable across all studied models and all layers. Qualitatively, we show that the descriptions produced by LatentLens are semantically meaningful and provide more fine-grained interpretations for humans compared to individual tokens. More broadly, our findings contribute new evidence on the alignment between vision and language representations, opening up new directions for analyzing latent representations.
698b928c6052d3bed963087f
Benno Krojer
claimed_verified
2026-02-11T11:15:40.382Z
6270c58780d5f35f8dbe42be
/avatars/d4d6e5eadfe9b1f47bce1c66728b24fc.svg
Benno Krojer
BennoKrojer
user
BennoKrojer
698b928c6052d3bed9630880
Shravan Nayak
claimed_verified
2026-03-17T09:37:55.992Z
643ee7606d5be535d28034f6
https://cdn-avatars.huggingface.co/v1/production/uploads/643ee7606d5be535d28034f6/1F241mP_buW74byC2GAWH.jpeg
Shravan Nayak
BAJUKA
user
BAJUKA
698b928c6052d3bed9630881
Oscar Mañas
698b928c6052d3bed9630882
Vaibhav Adlakha
698b928c6052d3bed9630883
Desmond Elliott
698b928c6052d3bed9630884
Siva Reddy
698b928c6052d3bed9630885
Marius Mosbach
LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs
Transforming a large language model (LLM) into a Vision-Language Model (VLM) can be achieved by mapping the visual tokens from a vision encoder into the embedding space of an LLM. Intriguingly, this mapping can be as simple as a shallow MLP transformation. To understand why LLMs can so readily process visual tokens, we need interpretability methods that reveal what is encoded in the visual token representations at every layer of LLM processing. In this work, we introduce LatentLens, a novel approach for mapping latent representations to descriptions in natural language. LatentLens works by encoding a large text corpus and storing contextualized token representations for each token in that corpus. Visual token representations are then compared to their contextualized textual representations, with the top-k nearest neighbor representations providing descriptions of the visual token. We evaluate this method on 10 different VLMs, showing that commonly used methods, such as LogitLens, substantially underestimate the interpretability of visual tokens. With LatentLens instead, the majority of visual tokens are interpretable across all studied models and all layers. Qualitatively, we show that the descriptions produced by LatentLens are semantically meaningful and provide more fine-grained interpretations for humans compared to individual tokens. More broadly, our findings contribute new evidence on the alignment between vision and language representations, opening up new directions for analyzing latent representations.
LatentLens enables interpretation of visual token representations in vision-language models by comparing them to contextualized textual representations, revealing that visual tokens are more interpretable than previously thought.
Vision-Language Model visual tokens embedding space LLM MLP transformation interpretability methods latent representations contextualized token representations nearest neighbor representations