Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representations
Harnessing the full potential of visually-rich documents requires retrieval systems that understand not just text, but intricate layouts, a core challenge in Visual Document Retrieval (VDR). The prevailing multi-vector architectures, while powerful, face a crucial storage bottleneck that current optimization strategies, such as embedding merging, pruning, or using abstract tokens, fail to resolve without compromising performance or ignoring vital layout cues. To address this, we introduce ColParse, a novel paradigm that leverages a document parsing model to generate a small set of layout-informed sub-image embeddings, which are then fused with a global page-level vector to create a compact and structurally-aware multi-vector representation. Extensive experiments demonstrate that our method reduces storage requirements by over 95% while simultaneously yielding significant performance gains across numerous benchmarks and base models. ColParse thus bridges the critical gap between the fine-grained accuracy of multi-vector retrieval and the practical demands of large-scale deployment, offering a new path towards efficient and interpretable multimodal information systems.
69ae265b9893f58272f74073
Yibo Yan
claimed_verified
2026-03-09T08:48:05.583Z
668def32af57d6e4b5a6d703
/avatars/508bd9eb2e9af1af0febc934f438f33f.svg
yiboyan
Buluchacha
user
Buluchacha
69ae265b9893f58272f74074
Mingdong Ou
69ae265b9893f58272f74075
Yi Cao
claimed_verified
2026-03-19T13:04:06.012Z
6344d95e762379fc6303b861
/avatars/4cd7e4adeccec72acb221b8b40fb95ab.svg
cy
frozenc
user
frozenc
69ae265b9893f58272f74076
Xin Zou
69ae265b9893f58272f74077
Shuliang Liu
69ae265b9893f58272f74078
Jiahao Huo
69ae265b9893f58272f74079
Yu Huang
69ae265b9893f58272f7407a
James Kwok
69ae265b9893f58272f7407b
Xuming Hu
Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representations
Harnessing the full potential of visually-rich documents requires retrieval systems that understand not just text, but intricate layouts, a core challenge in Visual Document Retrieval (VDR). The prevailing multi-vector architectures, while powerful, face a crucial storage bottleneck that current optimization strategies, such as embedding merging, pruning, or using abstract tokens, fail to resolve without compromising performance or ignoring vital layout cues. To address this, we introduce ColParse, a novel paradigm that leverages a document parsing model to generate a small set of layout-informed sub-image embeddings, which are then fused with a global page-level vector to create a compact and structurally-aware multi-vector representation. Extensive experiments demonstrate that our method reduces storage requirements by over 95% while simultaneously yielding significant performance gains across numerous benchmarks and base models. ColParse thus bridges the critical gap between the fine-grained accuracy of multi-vector retrieval and the practical demands of large-scale deployment, offering a new path towards efficient and interpretable multimodal information systems.
ColParse introduces a document parsing approach that generates layout-informed sub-image embeddings to create compact, structurally-aware representations for visual document retrieval, achieving over 95% storage reduction while maintaining performance.
Visual Document Retrieval multi-vector architectures embedding merging pruning abstract tokens document parsing model layout-informed sub-image embeddings global page-level vector compact representation structurally-aware multi-vector representation