LPM 1.0: Video-based Character Performance Model
Performance, the externalization of intent, emotion, and personality through visual, vocal, and temporal behavior, is what makes a character alive. Learning such performance from video is a promising alternative to traditional 3D pipelines. However, existing video models struggle to jointly achieve high expressiveness, real-time inference, and long-horizon identity stability, a tension we call the performance trilemma. Conversation is the most comprehensive performance scenario, as characters simultaneously speak, listen, react, and emote while maintaining identity over time. To address this, we present LPM 1.0 (Large Performance Model), focusing on single-person full-duplex audio-visual conversational performance. Concretely, we build a multimodal human-centric dataset through strict filtering, speaking-listening audio-video pairing, performance understanding, and identity-aware multi-reference extraction; train a 17B-parameter Diffusion Transformer (Base LPM) for highly controllable, identity-consistent performance through multimodal conditioning; and distill it into a causal streaming generator (Online LPM) for low-latency, infinite-length interaction. At inference, given a character image with identity-aware references, LPM 1.0 generates listening videos from user audio and speaking videos from synthesized audio, with text prompts for motion control, all at real-time speed with identity-stable, infinite-length generation. LPM 1.0 thus serves as a visual engine for conversational agents, live streaming characters, and game NPCs. To systematically evaluate this setting, we propose LPM-Bench, the first benchmark for interactive character performance. LPM 1.0 achieves state-of-the-art results across all evaluated dimensions while maintaining real-time inference.
69d8611679548974da946540
Ailing Zeng
69d8611679548974da946541
Casper Yang
69d8611679548974da946542
Chauncey Ge
69d8611679548974da946543
Eddie Zhang
claimed_verified
2026-04-10T07:43:14.848Z
61f24cbb88b9b5abbe184a85
https://cdn-avatars.huggingface.co/v1/production/uploads/61f24cbb88b9b5abbe184a85/OvcJRU51yI8pdO77NBHLb.jpeg
zhangfan
Fan-s
user
Fan-s
69d8611679548974da946544
Garvey Xu
69d8611679548974da946545
Gavin Lin
69d8611679548974da946546
Gilbert Gu
69d8611679548974da946547
Jeremy Pi
69d8611679548974da946548
Leo Li
69d8611679548974da946549
Mingyi Shi
69d8611679548974da94654a
Sheng Bi
69d8611679548974da94654b
Steven Tang
69d8611679548974da94654c
Thorn Hang
69d8611679548974da94654d
Tobey Guo
69d8611679548974da94654e
Vincent Li
69d8611679548974da94654f
Xin Tong
69d8611679548974da946550
Yikang Li
69d8611679548974da946551
Yuchen Sun
69d8611679548974da946552
Yue
69d8611679548974da946553
Zhao
claimed_verified
2026-04-13T07:33:38.023Z
641cc612f0b71a9743668604
/avatars/ab7d91df8de266fadb4e5632a1f0a37a.svg
Yue (R) Zhao
yzhao30
user
yzhao30
69d8611679548974da946554
Yuhan Lu
69d8611679548974da946555
Yuwei Li
69d8611679548974da946556
Zane Zhang
69d8611679548974da946557
Zeshi Yang
69d8611679548974da946558
Zi Ye
LPM 1.0: Video-based Character Performance Model
Performance, the externalization of intent, emotion, and personality through visual, vocal, and temporal behavior, is what makes a character alive. Learning such performance from video is a promising alternative to traditional 3D pipelines. However, existing video models struggle to jointly achieve high expressiveness, real-time inference, and long-horizon identity stability, a tension we call the performance trilemma. Conversation is the most comprehensive performance scenario, as characters simultaneously speak, listen, react, and emote while maintaining identity over time. To address this, we present LPM 1.0 (Large Performance Model), focusing on single-person full-duplex audio-visual conversational performance. Concretely, we build a multimodal human-centric dataset through strict filtering, speaking-listening audio-video pairing, performance understanding, and identity-aware multi-reference extraction; train a 17B-parameter Diffusion Transformer (Base LPM) for highly controllable, identity-consistent performance through multimodal conditioning; and distill it into a causal streaming generator (Online LPM) for low-latency, infinite-length interaction. At inference, given a character image with identity-aware references, LPM 1.0 generates listening videos from user audio and speaking videos from synthesized audio, with text prompts for motion control, all at real-time speed with identity-stable, infinite-length generation. LPM 1.0 thus serves as a visual engine for conversational agents, live streaming characters, and game NPCs. To systematically evaluate this setting, we propose LPM-Bench, the first benchmark for interactive character performance. LPM 1.0 achieves state-of-the-art results across all evaluated dimensions while maintaining real-time inference.
A large-scale multimodal model for real-time conversational character performance generation that maintains identity consistency while enabling interactive, infinite-length video synthesis.
Diffusion Transformer multimodal conditioning causal streaming generator identity-consistent performance audio-visual conversational performance real-time inference infinite-length generation visual engine LPM-Bench