SocialVeil: Probing Social Intelligence of Language Agents under Communication Barriers
Large language models (LLMs) are increasingly evaluated in interactive environments to test their social intelligence. However, existing benchmarks often assume idealized communication between agents, limiting our ability to diagnose whether LLMs can maintain and repair interactions in more realistic, imperfect settings. To close this gap, we present SocialVeil, a social learning environment that can simulate social interaction under cognitive-difference-induced communication barriers. Grounded in a systematic literature review of communication challenges in human interaction, SocialVeil introduces three representative types of such disruption, semantic vagueness, sociocultural mismatch, and emotional interference. We also introduce two barrier-aware evaluation metrics, unresolved confusion and mutual understanding, to evaluate interaction quality under impaired communication. Experiments across 720 scenarios and four frontier LLMs show that barriers consistently impair performance, with mutual understanding reduced by over 45\% on average, and confusion elevated by nearly 50\%. Human evaluations validate the fidelity of these simulated barriers (ICCapprox0.78, Pearson rapprox0.80). We further demonstrate that adaptation strategies (Repair Instruction and Interactive learning) only have a modest effect far from barrier-free performance. This work takes a step toward bringing social interaction environments closer to real-world communication, opening opportunities for exploring the social intelligence of LLM agents.
698619f59c78be977c104bac
Keyang Xuan
claimed_verified
2026-02-09T08:34:55.860Z
687ebb87416fcff56959d817
/avatars/8f13090dd6179bb18e8c8d205fd20131.svg
Keyang Xuan
keyangx3
user
keyangx3
698619f59c78be977c104bad
Pengda Wang
698619f59c78be977c104bae
Chongrui Ye
698619f59c78be977c104baf
Haofei Yu
698619f59c78be977c104bb0
Tal August
698619f59c78be977c104bb1
Jiaxuan You
SocialVeil: Probing Social Intelligence of Language Agents under Communication Barriers
Large language models (LLMs) are increasingly evaluated in interactive environments to test their social intelligence. However, existing benchmarks often assume idealized communication between agents, limiting our ability to diagnose whether LLMs can maintain and repair interactions in more realistic, imperfect settings. To close this gap, we present SocialVeil, a social learning environment that can simulate social interaction under cognitive-difference-induced communication barriers. Grounded in a systematic literature review of communication challenges in human interaction, SocialVeil introduces three representative types of such disruption, semantic vagueness, sociocultural mismatch, and emotional interference. We also introduce two barrier-aware evaluation metrics, unresolved confusion and mutual understanding, to evaluate interaction quality under impaired communication. Experiments across 720 scenarios and four frontier LLMs show that barriers consistently impair performance, with mutual understanding reduced by over 45\% on average, and confusion elevated by nearly 50\%. Human evaluations validate the fidelity of these simulated barriers (ICCapprox0.78, Pearson rapprox0.80). We further demonstrate that adaptation strategies (Repair Instruction and Interactive learning) only have a modest effect far from barrier-free performance. This work takes a step toward bringing social interaction environments closer to real-world communication, opening opportunities for exploring the social intelligence of LLM agents.
SocialVeil presents a social learning environment that simulates communication barriers in LLM interactions, demonstrating significant performance degradation under realistic conditions and limited effectiveness of adaptation strategies.
large language models social intelligence communication barriers semantic vagueness sociocultural mismatch emotional interference unresolved confusion mutual understanding interactive learning repair instruction