PEARL: Personalized Streaming Video Understanding Model
Human cognition of new concepts is inherently a streaming process: we continuously recognize new objects or identities and update our memories over time. However, current multimodal personalization methods are largely limited to static images or offline videos. This disconnects continuous visual input from instant real-world feedback, limiting their ability to provide the real-time, interactive personalized responses essential for future AI assistants. To bridge this gap, we first propose and formally define the novel task of Personalized Streaming Video Understanding (PSVU). To facilitate research in this new direction, we introduce PEARL-Bench, the first comprehensive benchmark designed specifically to evaluate this challenging setting. It evaluates a model's ability to respond to personalized concepts at exact timestamps under two modes: (1) Frame-level, focusing on a specific person or object in discrete frames, and (2) a novel Video-level, focusing on personalized actions unfolding across continuous frames. PEARL-Bench comprises 132 unique videos and 2,173 fine-grained annotations with precise timestamps. Concept diversity and annotation quality are strictly ensured through a combined pipeline of automated generation and human verification. To tackle this challenging new setting, we further propose PEARL, a plug-and-play, training-free strategy that serves as a strong baseline. Extensive evaluations across 8 offline and online models demonstrate that PEARL achieves state-of-the-art performance. Notably, it brings consistent PSVU improvements when applied to 3 distinct architectures, proving to be a highly effective and robust strategy. We hope this work advances vision-language model (VLM) personalization and inspires further research into streaming personalized AI assistants. Code is available at https://github.com/Yuanhong-Zheng/PEARL.
69c205e3c105f45cb0481ede
Yuanhong Zheng
claimed_verified
2026-03-24T09:21:42.487Z
6537e7e55ad6715c4c43297b
/avatars/069e4afb7efdbef0c467461e8d390bc9.svg
zhengyuanhong
zyh200727
user
zyh200727
69c205e3c105f45cb0481edf
Ruichuan An
69c205e3c105f45cb0481ee0
Xiaopeng Lin
69c205e3c105f45cb0481ee1
Yuxing Liu
69c205e3c105f45cb0481ee2
Sihan Yang
69c205e3c105f45cb0481ee3
Huanyu Zhang
69c205e3c105f45cb0481ee4
Haodong Li
claimed_verified
2026-03-24T13:21:02.650Z
65ddea8b2d26e59a5a33330f
/avatars/3104ddafd6dda3c05ea9a771dbf2deeb.svg
li haodong
mickyhimself
user
mickyhimself
69c205e3c105f45cb0481ee5
Qintong Zhang
69c205e3c105f45cb0481ee6
Renrui Zhang
69c205e3c105f45cb0481ee7
Guopeng Li
69c205e3c105f45cb0481ee8
Yifan Zhang
69c205e3c105f45cb0481ee9
Yuheng Li
69c205e3c105f45cb0481eea
Wentao Zhang
PEARL: Personalized Streaming Video Understanding Model
Human cognition of new concepts is inherently a streaming process: we continuously recognize new objects or identities and update our memories over time. However, current multimodal personalization methods are largely limited to static images or offline videos. This disconnects continuous visual input from instant real-world feedback, limiting their ability to provide the real-time, interactive personalized responses essential for future AI assistants. To bridge this gap, we first propose and formally define the novel task of Personalized Streaming Video Understanding (PSVU). To facilitate research in this new direction, we introduce PEARL-Bench, the first comprehensive benchmark designed specifically to evaluate this challenging setting. It evaluates a model's ability to respond to personalized concepts at exact timestamps under two modes: (1) Frame-level, focusing on a specific person or object in discrete frames, and (2) a novel Video-level, focusing on personalized actions unfolding across continuous frames. PEARL-Bench comprises 132 unique videos and 2,173 fine-grained annotations with precise timestamps. Concept diversity and annotation quality are strictly ensured through a combined pipeline of automated generation and human verification. To tackle this challenging new setting, we further propose PEARL, a plug-and-play, training-free strategy that serves as a strong baseline. Extensive evaluations across 8 offline and online models demonstrate that PEARL achieves state-of-the-art performance. Notably, it brings consistent PSVU improvements when applied to 3 distinct architectures, proving to be a highly effective and robust strategy. We hope this work advances vision-language model (VLM) personalization and inspires further research into streaming personalized AI assistants. Code is available at https://github.com/Yuanhong-Zheng/PEARL.
Personalized streaming video understanding addresses real-time visual input processing with precise temporal annotations, enabling interactive AI assistants through a new benchmark and plug-and-play strategy.
multimodal personalization streaming process visual input real-time feedback personalized streaming video understanding PEARL-Bench frame-level video-level vision-language model training-free strategy plug-and-play