SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models
Omni-modal large language models (OLMs) redefine human-machine interaction by natively integrating audio, vision, and text. However, existing OLM benchmarks remain anchored to static, accuracy-centric tasks, leaving a critical gap in assessing social interactivity, the fundamental capacity to navigate dynamic cues in natural dialogues. To this end, we propose SocialOmni, a comprehensive benchmark that operationalizes the evaluation of this conversational interactivity across three core dimensions: (i) speaker separation and identification (who is speaking), (ii) interruption timing control (when to interject), and (iii) natural interruption generation (how to phrase the interruption). SocialOmni features 2,000 perception samples and a quality-controlled diagnostic set of 209 interaction-generation instances with strict temporal and contextual constraints, complemented by controlled audio-visual inconsistency scenarios to test model robustness. We benchmarked 12 leading OLMs, which uncovers significant variance in their social-interaction capabilities across models. Furthermore, our analysis reveals a pronounced decoupling between a model's perceptual accuracy and its ability to generate contextually appropriate interruptions, indicating that understanding-centric metrics alone are insufficient to characterize conversational social competence. More encouragingly, these diagnostics from SocialOmni yield actionable signals for bridging the perception-interaction divide in future OLMs.
69ba0c3c5d2b9c28ea09f04b
Tianyu Xie
admin_assigned
2026-03-23T11:26:13.207Z
65a7b8bd0dba6b99a094e527
/avatars/d6e3797fef3e7c96114fa2ccd84f73a6.svg
Tianyu Xie
alexisty
user
alexisty
69ba0c3c5d2b9c28ea09f04c
Jinfa Huang
claimed_verified
2026-03-18T08:45:07.953Z
63f37af60be81bdc5d92eebb
/avatars/b8dfdff4ab36988ec9a8643e82a3d2db.svg
Huang
Jinfa
user
Jinfa
69ba0c3c5d2b9c28ea09f04d
Yuexiao Ma
admin_assigned
2026-03-18T10:04:32.362Z
65223ebca692c870eab39e94
/avatars/13f40c0a0a747795d7f12455d1700394.svg
yuexiao Ma
bobxmuma
user
bobxmuma
69ba0c3c5d2b9c28ea09f04e
Rongfang Luo
admin_assigned
2026-03-18T10:04:38.962Z
69ba4086c6774a70397e6ae0
/avatars/f1a3919cbf469a4712390597dc4a348a.svg
RongFang Luo
lrf-502
user
lrf-502
69ba0c3c5d2b9c28ea09f04f
Yan Yang
69ba0c3c5d2b9c28ea09f050
Wang Chen
69ba0c3c5d2b9c28ea09f051
Yuhui Zeng
admin_assigned
2026-03-18T10:04:46.979Z
67ce1014015315093a551712
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/qUWLSiCfOcNYzDyv2Alj_.png
YUHUI ZENG
YuhuiZeng
user
YuhuiZeng
69ba0c3c5d2b9c28ea09f052
Ruize Fang
admin_assigned
2026-03-18T10:04:53.832Z
68a7270e4e2a935986ccea53
/avatars/0cc376026d543846db23b5b01163030e.svg
Ruize Fang
frz2005
user
frz2005
69ba0c3c5d2b9c28ea09f053
Yixuan Zou
69ba0c3c5d2b9c28ea09f054
Xiawu Zheng
69ba0c3c5d2b9c28ea09f055
Jiebo Luo
69ba0c3c5d2b9c28ea09f056
Rongrong Ji
SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models
Omni-modal large language models (OLMs) redefine human-machine interaction by natively integrating audio, vision, and text. However, existing OLM benchmarks remain anchored to static, accuracy-centric tasks, leaving a critical gap in assessing social interactivity, the fundamental capacity to navigate dynamic cues in natural dialogues. To this end, we propose SocialOmni, a comprehensive benchmark that operationalizes the evaluation of this conversational interactivity across three core dimensions: (i) speaker separation and identification (who is speaking), (ii) interruption timing control (when to interject), and (iii) natural interruption generation (how to phrase the interruption). SocialOmni features 2,000 perception samples and a quality-controlled diagnostic set of 209 interaction-generation instances with strict temporal and contextual constraints, complemented by controlled audio-visual inconsistency scenarios to test model robustness. We benchmarked 12 leading OLMs, which uncovers significant variance in their social-interaction capabilities across models. Furthermore, our analysis reveals a pronounced decoupling between a model's perceptual accuracy and its ability to generate contextually appropriate interruptions, indicating that understanding-centric metrics alone are insufficient to characterize conversational social competence. More encouragingly, these diagnostics from SocialOmni yield actionable signals for bridging the perception-interaction divide in future OLMs.
SocialOmni presents a benchmark for evaluating social interactivity in omni-modal large language models across speaker identification, interruption timing, and natural interruption generation, revealing gaps between perceptual accuracy and conversational competence.
omni-modal large language models social interactivity speaker separation interruption timing natural interruption generation perception samples interaction-generation instances audio-visual inconsistency scenarios conversational social competence