Doc-PP: Document Policy Preservation Benchmark for Large Vision-Language Models
The deployment of Large Vision-Language Models (LVLMs) for real-world document question answering is often constrained by dynamic, user-defined policies that dictate information disclosure based on context. While ensuring adherence to these explicit constraints is critical, existing safety research primarily focuses on implicit social norms or text-only settings, overlooking the complexities of multimodal documents. In this paper, we introduce Doc-PP (Document Policy Preservation Benchmark), a novel benchmark constructed from real-world reports requiring reasoning across heterogeneous visual and textual elements under strict non-disclosure policies. Our evaluation highlights a systemic Reasoning-Induced Safety Gap: models frequently leak sensitive information when answers must be inferred through complex synthesis or aggregated across modalities, effectively circumventing existing safety constraints. Furthermore, we identify that providing extracted text improves perception but inadvertently facilitates leakage. To address these vulnerabilities, we propose DVA (Decompose-Verify-Aggregation), a structural inference framework that decouples reasoning from policy verification. Experimental results demonstrate that DVA significantly outperforms standard prompting defenses, offering a robust baseline for policy-compliant document understanding
695f10e05fa3847525c41cee
Haeun Jang
695f10e05fa3847525c41cef
Hwan Chang
claimed_verified
2026-01-08T08:32:40.562Z
647c4a2692182942d7c2e698
/avatars/bcddf5fe49aa092a2645f70812108348.svg
HWANCHANG
HwanChang0106
user
HwanChang0106
695f10e05fa3847525c41cf0
Hwanhee Lee
Doc-PP: Document Policy Preservation Benchmark for Large Vision-Language Models
The deployment of Large Vision-Language Models (LVLMs) for real-world document question answering is often constrained by dynamic, user-defined policies that dictate information disclosure based on context. While ensuring adherence to these explicit constraints is critical, existing safety research primarily focuses on implicit social norms or text-only settings, overlooking the complexities of multimodal documents. In this paper, we introduce Doc-PP (Document Policy Preservation Benchmark), a novel benchmark constructed from real-world reports requiring reasoning across heterogeneous visual and textual elements under strict non-disclosure policies. Our evaluation highlights a systemic Reasoning-Induced Safety Gap: models frequently leak sensitive information when answers must be inferred through complex synthesis or aggregated across modalities, effectively circumventing existing safety constraints. Furthermore, we identify that providing extracted text improves perception but inadvertently facilitates leakage. To address these vulnerabilities, we propose DVA (Decompose-Verify-Aggregation), a structural inference framework that decouples reasoning from policy verification. Experimental results demonstrate that DVA significantly outperforms standard prompting defenses, offering a robust baseline for policy-compliant document understanding
Large Vision-Language Models face challenges in document question answering due to policy compliance issues, leading to the introduction of a new benchmark and a structural inference framework to enhance safety and prevent information leakage.
Large Vision-Language Models Document Policy Preservation Benchmark Reasoning-Induced Safety Gap Decompose-Verify-Aggregation policy compliance information leakage multimodal documents visual reasoning textual elements