A Systematic Study of Cross-Modal Typographic Attacks on Audio-Visual Reasoning
As audio-visual multi-modal large language models (MLLMs) are increasingly deployed in safety-critical applications, understanding their vulnerabilities is crucial. To this end, we introduce Multi-Modal Typography, a systematic study examining how typographic attacks across multiple modalities adversely influence MLLMs. While prior work focuses narrowly on unimodal attacks, we expose the cross-modal fragility of MLLMs. We analyze the interactions between audio, visual, and text perturbations and reveal that coordinated multi-modal attack creates a significantly more potent threat than single-modality attacks (attack success rate = 83.43% vs 34.93%).Our findings across multiple frontier MLLMs, tasks, and common-sense reasoning and content moderation benchmarks establishes multi-modal typography as a critical and underexplored attack strategy in multi-modal reasoning. Code and data will be publicly available.
69d7761ea3ce5cf29b5e744e
Tianle Chen
69d7761ea3ce5cf29b5e744f
Deepti Ghadiyaram
claimed_verified
2026-04-13T07:34:14.972Z
6905068e4c9138632af2fa21
/avatars/c32215347601de938c218e6c81dc2e33.svg
Deepti Ghadiyaram
dghadiya
user
dghadiya
A Systematic Study of Cross-Modal Typographic Attacks on Audio-Visual Reasoning
As audio-visual multi-modal large language models (MLLMs) are increasingly deployed in safety-critical applications, understanding their vulnerabilities is crucial. To this end, we introduce Multi-Modal Typography, a systematic study examining how typographic attacks across multiple modalities adversely influence MLLMs. While prior work focuses narrowly on unimodal attacks, we expose the cross-modal fragility of MLLMs. We analyze the interactions between audio, visual, and text perturbations and reveal that coordinated multi-modal attack creates a significantly more potent threat than single-modality attacks (attack success rate = 83.43% vs 34.93%).Our findings across multiple frontier MLLMs, tasks, and common-sense reasoning and content moderation benchmarks establishes multi-modal typography as a critical and underexplored attack strategy in multi-modal reasoning. Code and data will be publicly available.
Multi-modal typography attacks demonstrate significantly higher success rates than unimodal attacks by exploiting cross-modal vulnerabilities in audio-visual multi-modal large language models.
audio-visual multi-modal large language models typographic attacks cross-modal fragility multi-modal attack unimodal attacks multi-modal reasoning content moderation benchmarks common-sense reasoning