PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation Agents
Current Graphical User Interface (GUI) agents operate primarily under a reactive paradigm: a user must provide an explicit instruction for the agent to execute a task. However, an intelligent AI assistant should be proactive, which is capable of anticipating user intentions directly from continuous visual inputs, such as mobile or desktop screenshots, and offering timely recommendations without explicit user prompting. Transitioning to this proactive paradigm presents significant challenges. Real-world screen activity is rarely linear; it consists of long-horizon trajectories fraught with noisy browsing, meaningless actions, and multithreaded task-switching. To address this gap, we introduce PIRA-Bench (Proactive Intent Recommendation Agent Benchmark), a novel benchmark for evaluating multimodal large language models (MLLMs) on continuous, weakly-supervised visual inputs. Unlike reactive datasets, PIRA-Bench features complex trajectories with multiple interleaved intents and noisy segments with various user profile contexts, challenging agents to detect actionable events while fitting to user preferences. Furthermore, we propose the PIRF baseline, a memory-aware, state-tracking framework that empowers general MLLMs to manage multiple task threads and handle misleading visual inputs. PIRA-Bench serves as an initial step toward robust and proactive GUI-based personal assistants.
69af7f00b3ab31dee1281bf2
Yuxiang Chai
claimed_verified
2026-03-10T09:45:20.020Z
6458ce236fa580137af5aa95
/avatars/db65a7332e375eb5daad5c1b076b1e3b.svg
Yuxiang Chai
Yuxiang007
user
Yuxiang007
69af7f00b3ab31dee1281bf3
Shunye Tang
69af7f00b3ab31dee1281bf4
Han Xiao
claimed_verified
2026-03-10T09:45:22.142Z
666aa99cd1652853e4f9a8b9
/avatars/7cd5a0c34b5ccb8eff5a353d88d15a93.svg
HanXiao
HanXiao1999
user
HanXiao1999
69af7f00b3ab31dee1281bf5
Rui Liu
69af7f00b3ab31dee1281bf6
Hongsheng Li
PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation Agents
Current Graphical User Interface (GUI) agents operate primarily under a reactive paradigm: a user must provide an explicit instruction for the agent to execute a task. However, an intelligent AI assistant should be proactive, which is capable of anticipating user intentions directly from continuous visual inputs, such as mobile or desktop screenshots, and offering timely recommendations without explicit user prompting. Transitioning to this proactive paradigm presents significant challenges. Real-world screen activity is rarely linear; it consists of long-horizon trajectories fraught with noisy browsing, meaningless actions, and multithreaded task-switching. To address this gap, we introduce PIRA-Bench (Proactive Intent Recommendation Agent Benchmark), a novel benchmark for evaluating multimodal large language models (MLLMs) on continuous, weakly-supervised visual inputs. Unlike reactive datasets, PIRA-Bench features complex trajectories with multiple interleaved intents and noisy segments with various user profile contexts, challenging agents to detect actionable events while fitting to user preferences. Furthermore, we propose the PIRF baseline, a memory-aware, state-tracking framework that empowers general MLLMs to manage multiple task threads and handle misleading visual inputs. PIRA-Bench serves as an initial step toward robust and proactive GUI-based personal assistants.
PIRA-Bench presents a benchmark for evaluating multimodal large language models on proactive GUI agent tasks using continuous visual inputs, while PIRF offers a memory-aware framework for handling complex, noisy screen trajectories.
multimodal large language models visual inputs proactive GUI agents continuous visual inputs weakly-supervised learning task threads state-tracking framework multimodal large language models