PLUME: Latent Reasoning Based Universal Multimodal Embedding
Universal multimodal embedding (UME) maps heterogeneous inputs into a shared retrieval space with a single model. Recent approaches improve UME by generating explicit chain-of-thought (CoT) rationales before extracting embeddings, enabling multimodal large language models to better infer complex query intent. However, explicit CoT incurs substantial inference overhead and can compress rich multimodal evidence into a narrow textual bottleneck. We propose PLUME, a latent reasoning framework that advances UME by replacing verbalized CoT with a short autoregressive rollout of continuous latent states. To support diverse multimodal queries, PLUME further introduces a semantic-anchor-guided transition adapter that steers latent rollout along different reasoning trajectories under the same fixed computation budget. To stabilize training, PLUME adopts a progressive explicit-to-latent curriculum that uses verbalized reasoning only as a temporary training scaffold and gradually transfers this behavior into hidden-state computation, eliminating explicit CoT at inference. On the 78-task MMEB-v2 benchmark, PLUME outperforms strong explicit-CoT UME baselines while reducing reasoning from hundreds of generated tokens to fewer than 10 latent steps, delivering over 30x faster inference. PLUME is especially well suited to retrieval settings where relevant evidence is dense, structurally complex, and difficult to organize through verbalized intermediate rationales, such as video and visual document retrieval. These results show that structured latent computation can preserve the benefits of intermediate reasoning without the overhead of explicit rationale generation, providing a stronger and more efficient paradigm for practical retrieval systems.
69d4c70c2c523d36431fb418
Chenwei He
69d4c70c2c523d36431fb419
Xiangzhao Hao
claimed_verified
2026-04-07T13:33:54.276Z
64ec6025b96ff0e175728ac0
/avatars/a1acbe22b5e5105703e7912da2cfced2.svg
hxz
CUDAOUTOFMEMORY
user
CUDAOUTOFMEMORY
69d4c70c2c523d36431fb41a
Tianyu Yang
69d4c70c2c523d36431fb41b
Yuxiang Ma
69d4c70c2c523d36431fb41c
Yuheng Jia
69d4c70c2c523d36431fb41d
Lingxiang Wu
69d4c70c2c523d36431fb41e
Chaoyang Zhao
69d4c70c2c523d36431fb41f
Haiyun Guo
69d4c70c2c523d36431fb420
Jinqiao Wang
PLUME: Latent Reasoning Based Universal Multimodal Embedding
Universal multimodal embedding (UME) maps heterogeneous inputs into a shared retrieval space with a single model. Recent approaches improve UME by generating explicit chain-of-thought (CoT) rationales before extracting embeddings, enabling multimodal large language models to better infer complex query intent. However, explicit CoT incurs substantial inference overhead and can compress rich multimodal evidence into a narrow textual bottleneck. We propose PLUME, a latent reasoning framework that advances UME by replacing verbalized CoT with a short autoregressive rollout of continuous latent states. To support diverse multimodal queries, PLUME further introduces a semantic-anchor-guided transition adapter that steers latent rollout along different reasoning trajectories under the same fixed computation budget. To stabilize training, PLUME adopts a progressive explicit-to-latent curriculum that uses verbalized reasoning only as a temporary training scaffold and gradually transfers this behavior into hidden-state computation, eliminating explicit CoT at inference. On the 78-task MMEB-v2 benchmark, PLUME outperforms strong explicit-CoT UME baselines while reducing reasoning from hundreds of generated tokens to fewer than 10 latent steps, delivering over 30x faster inference. PLUME is especially well suited to retrieval settings where relevant evidence is dense, structurally complex, and difficult to organize through verbalized intermediate rationales, such as video and visual document retrieval. These results show that structured latent computation can preserve the benefits of intermediate reasoning without the overhead of explicit rationale generation, providing a stronger and more efficient paradigm for practical retrieval systems.
PLUME introduces a latent reasoning framework for universal multimodal embedding that replaces explicit chain-of-thought reasoning with continuous latent state rollouts, achieving faster inference while maintaining performance on complex multimodal retrieval tasks.
universal multimodal embedding chain-of-thought latent reasoning autoregressive rollout continuous latent states semantic-anchor-guided transition adapter progressive explicit-to-latent curriculum multimodal large language models retrieval space MMEB-v2 benchmark