Proact-VL: A Proactive VideoLLM for Real-Time AI Companions
Proactive and real-time interactive experiences are essential for human-like AI companions, yet face three key challenges: (1) achieving low-latency inference under continuous streaming inputs, (2) autonomously deciding when to respond, and (3) controlling both quality and quantity of generated content to meet real-time constraints. In this work, we instantiate AI companions through two gaming scenarios, commentator and guide, selected for their suitability for automatic evaluation. We introduce the Live Gaming Benchmark, a large-scale dataset with three representative scenarios: solo commentary, co-commentary, and user guidance, and present Proact-VL, a general framework that shapes multimodal language models into proactive, real-time interactive agents capable of human-like environment perception and interaction. Extensive experiments show Proact-VL achieves superior response latency and quality while maintaining strong video understanding capabilities, demonstrating its practicality for real-time interactive applications.
69a900118b5b27c7ea9fa674
Weicai Yan
admin_assigned
2026-03-05T10:26:08.229Z
64aa816351756fb15c0f6473
https://cdn-avatars.huggingface.co/v1/production/uploads/64aa816351756fb15c0f6473/x2jSWE7PPawNHndGes1cV.jpeg
Weicai Yan
oaaoaa
user
oaaoaa
69a900118b5b27c7ea9fa675
Yuhong Dai
claimed_verified
2026-03-05T10:02:01.127Z
65d70e775e971572da16c05b
https://cdn-avatars.huggingface.co/v1/production/uploads/65d70e775e971572da16c05b/8Cv71Clfk_C7k6U4yI6ln.jpeg
Yuhong Dai
BroAlanTaps
user
BroAlanTaps
69a900118b5b27c7ea9fa676
Qi Ran
69a900118b5b27c7ea9fa677
Haodong Li
claimed_verified
2026-03-05T10:02:03.357Z
65ddea8b2d26e59a5a33330f
/avatars/3104ddafd6dda3c05ea9a771dbf2deeb.svg
li haodong
mickyhimself
user
mickyhimself
69a900118b5b27c7ea9fa678
Wang Lin
69a900118b5b27c7ea9fa679
Hao Liao
69a900118b5b27c7ea9fa67a
Xing Xie
69a900118b5b27c7ea9fa67b
Tao Jin
69a900118b5b27c7ea9fa67c
Jianxun Lian
Proact-VL: A Proactive VideoLLM for Real-Time AI Companions
Proactive and real-time interactive experiences are essential for human-like AI companions, yet face three key challenges: (1) achieving low-latency inference under continuous streaming inputs, (2) autonomously deciding when to respond, and (3) controlling both quality and quantity of generated content to meet real-time constraints. In this work, we instantiate AI companions through two gaming scenarios, commentator and guide, selected for their suitability for automatic evaluation. We introduce the Live Gaming Benchmark, a large-scale dataset with three representative scenarios: solo commentary, co-commentary, and user guidance, and present Proact-VL, a general framework that shapes multimodal language models into proactive, real-time interactive agents capable of human-like environment perception and interaction. Extensive experiments show Proact-VL achieves superior response latency and quality while maintaining strong video understanding capabilities, demonstrating its practicality for real-time interactive applications.
Proact-VL is a multimodal framework that enables real-time interactive AI companions for gaming scenarios with low-latency responses and strong video understanding capabilities.
multimodal language models real-time interactive agents environment perception video understanding response latency proactive AI companions