LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety
Large language models (LLMs) often demonstrate strong safety performance in high-resource languages, yet exhibit severe vulnerabilities when queried in low-resource languages. We attribute this gap to a mismatch between language-agnostic semantic understanding ability and language-dominant safety alignment biased toward high-resource languages. Consistent with this hypothesis, we empirically identify the semantic bottleneck in LLMs, an intermediate layer in which the geometry of model representations is governed primarily by shared semantic content rather than language identity. Building on this observation, we propose Language-Agnostic Semantic Alignment (LASA), which anchors safety alignment directly in semantic bottlenecks. Experiments show that LASA substantially improves safety across all languages: average attack success rate (ASR) drops from 24.7% to 2.8% on LLaMA-3.1-8B-Instruct and remains around 3-4% across Qwen2.5 and Qwen3 Instruct models (7B-32B). Together, our analysis and method offer a representation-level perspective on LLM safety, suggesting that safety alignment requires anchoring safety understanding not in surface text, but in the model's language-agnostic semantic space.
69dfaefdf93805b185f6dbd6
Junxiao Yang
claimed_verified
2026-04-16T08:02:46.055Z
65d859a3661492b25c46a117
https://cdn-avatars.huggingface.co/v1/production/uploads/65d859a3661492b25c46a117/Yui5RSHsWltBF3s3X4NI2.jpeg
Junxiao Yang
yangjunxiao2021
user
yangjunxiao2021
69dfaefdf93805b185f6dbd7
Haoran Liu
69dfaefdf93805b185f6dbd8
Jinzhe Tu
69dfaefdf93805b185f6dbd9
Jiale Cheng
69dfaefdf93805b185f6dbda
Zhexin Zhang
69dfaefdf93805b185f6dbdb
Shiyao Cui
69dfaefdf93805b185f6dbdc
Jiaqi Weng
69dfaefdf93805b185f6dbdd
Jialing Tao
69dfaefdf93805b185f6dbde
Hui Xue
69dfaefdf93805b185f6dbdf
Hongning Wang
69dfaefdf93805b185f6dbe0
Han Qiu
69dfaefdf93805b185f6dbe1
Minlie Huang
LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety
Large language models (LLMs) often demonstrate strong safety performance in high-resource languages, yet exhibit severe vulnerabilities when queried in low-resource languages. We attribute this gap to a mismatch between language-agnostic semantic understanding ability and language-dominant safety alignment biased toward high-resource languages. Consistent with this hypothesis, we empirically identify the semantic bottleneck in LLMs, an intermediate layer in which the geometry of model representations is governed primarily by shared semantic content rather than language identity. Building on this observation, we propose Language-Agnostic Semantic Alignment (LASA), which anchors safety alignment directly in semantic bottlenecks. Experiments show that LASA substantially improves safety across all languages: average attack success rate (ASR) drops from 24.7% to 2.8% on LLaMA-3.1-8B-Instruct and remains around 3-4% across Qwen2.5 and Qwen3 Instruct models (7B-32B). Together, our analysis and method offer a representation-level perspective on LLM safety, suggesting that safety alignment requires anchoring safety understanding not in surface text, but in the model's language-agnostic semantic space.
Language-Agnostic Semantic Alignment (LASA) addresses LLM safety gaps across languages by targeting semantic bottlenecks where representations are primarily driven by shared semantics rather than language identity, achieving significant improvements in safety performance across multiple models.
large language models safety alignment semantic bottleneck language-agnostic representation-level perspective attack success rate LLaMA Qwen