ReactMotion: Generating Reactive Listener Motions from Speaker Utterance
In this paper, we introduce a new task, Reactive Listener Motion Generation from Speaker Utterance, which aims to generate naturalistic listener body motions that appropriately respond to a speaker's utterance. However, modeling such nonverbal listener behaviors remains underexplored and challenging due to the inherently non-deterministic nature of human reactions. To facilitate this task, we present ReactMotionNet, a large-scale dataset that pairs speaker utterances with multiple candidate listener motions annotated with varying degrees of appropriateness. This dataset design explicitly captures the one-to-many nature of listener behavior and provides supervision beyond a single ground-truth motion. Building on this dataset design, we develop preference-oriented evaluation protocols tailored to evaluate reactive appropriateness, where conventional motion metrics focusing on input-motion alignment ignore. We further propose ReactMotion, a unified generative framework that jointly models text, audio, emotion, and motion, and is trained with preference-based objectives to encourage both appropriate and diverse listener responses. Extensive experiments show that ReactMotion outperforms retrieval baselines and cascaded LLM-based pipelines, generating more natural, diverse, and appropriate listener motions.
69bd3ed56f6dd7c037056c02
Cheng Luo
claimed_verified
2026-03-23T10:51:21.806Z
631eb31f6d6a5870f3da46dc
/avatars/4855aafc740ad7cd99ceeb312cc2f56f.svg
Cheng
luochengzz
user
luochengzz
69bd3ed56f6dd7c037056c03
Bizhu Wu
69bd3ed56f6dd7c037056c04
Bing Li
claimed_verified
2026-03-23T10:51:19.705Z
666ddb45c0f3d5afc27e85ba
/avatars/dce98fc77bd8cf9e348f2d91bc3c0225.svg
Bing Li
bing-li-ai
user
bing-li-ai
69bd3ed56f6dd7c037056c05
Jianfeng Ren
69bd3ed56f6dd7c037056c06
Ruibin Bai
69bd3ed56f6dd7c037056c07
Rong Qu
69bd3ed56f6dd7c037056c08
Linlin Shen
69bd3ed56f6dd7c037056c09
Bernard Ghanem
ReactMotion: Generating Reactive Listener Motions from Speaker Utterance
In this paper, we introduce a new task, Reactive Listener Motion Generation from Speaker Utterance, which aims to generate naturalistic listener body motions that appropriately respond to a speaker's utterance. However, modeling such nonverbal listener behaviors remains underexplored and challenging due to the inherently non-deterministic nature of human reactions. To facilitate this task, we present ReactMotionNet, a large-scale dataset that pairs speaker utterances with multiple candidate listener motions annotated with varying degrees of appropriateness. This dataset design explicitly captures the one-to-many nature of listener behavior and provides supervision beyond a single ground-truth motion. Building on this dataset design, we develop preference-oriented evaluation protocols tailored to evaluate reactive appropriateness, where conventional motion metrics focusing on input-motion alignment ignore. We further propose ReactMotion, a unified generative framework that jointly models text, audio, emotion, and motion, and is trained with preference-based objectives to encourage both appropriate and diverse listener responses. Extensive experiments show that ReactMotion outperforms retrieval baselines and cascaded LLM-based pipelines, generating more natural, diverse, and appropriate listener motions.
A new task and framework are presented for generating listener body motions that react naturally to speaker utterances, using a large-scale dataset and preference-based training to produce diverse and appropriate responses.
Reactive Listener Motion Generation ReactMotionNet preference-oriented evaluation ReactMotion generative framework preference-based objectives