MA-EgoQA: Question Answering over Egocentric Videos from Multiple Embodied Agents
As embodied models become powerful, humans will collaborate with multiple embodied AI agents at their workplace or home in the future. To ensure better communication between human users and the multi-agent system, it is crucial to interpret incoming information from agents in parallel and refer to the appropriate context for each query. Existing challenges include effectively compressing and communicating high volumes of individual sensory inputs in the form of video and correctly aggregating multiple egocentric videos to construct system-level memory. In this work, we first formally define a novel problem of understanding multiple long-horizon egocentric videos simultaneously collected from embodied agents. To facilitate research in this direction, we introduce MultiAgent-EgoQA (MA-EgoQA), a benchmark designed to systemically evaluate existing models in our scenario. MA-EgoQA provides 1.7k questions unique to multiple egocentric streams, spanning five categories: social interaction, task coordination, theory-of-mind, temporal reasoning, and environmental interaction. We further propose a simple baseline model for MA-EgoQA named EgoMAS, which leverages shared memory across embodied agents and agent-wise dynamic retrieval. Through comprehensive evaluation across diverse baselines and EgoMAS on MA-EgoQA, we find that current approaches are unable to effectively handle multiple egocentric streams, highlighting the need for future advances in system-level understanding across the agents. The code and benchmark are available at https://ma-egoqa.github.io.
69b22e4d206bad330cd7ff88
Kangsan Kim
admin_assigned
2026-03-12T09:26:57.502Z
66ed7737f2f27a5dfd81ef09
/avatars/f45eea356e92ac7b3db23c2c92dec9fa.svg
Kangsan Kim
KangsanKim71
user
KangsanKim71
69b22e4d206bad330cd7ff89
Yanlai Yang
admin_assigned
2026-03-12T09:27:11.653Z
63334bf874ffb1a32c93e0a4
/avatars/4903c86bf71e96d80aafd87b80bd0a11.svg
Yanlai Yang
yanlaiy
user
yanlaiy
69b22e4d206bad330cd7ff8a
Suji Kim
admin_assigned
2026-03-12T09:27:17.947Z
65caff7e024c6320f4384130
/avatars/67f7c20372bfcc0c1554fb8c9284653a.svg
Suji Kim
SujiKim
user
SujiKim
69b22e4d206bad330cd7ff8b
Woongyeong Yeo
admin_assigned
2026-03-12T09:27:23.543Z
66d30f5fad293ffc4b7672bc
/avatars/6f164d813b947940a088820f8fd4dbe8.svg
Woongyeong Yeo
wgcyeo
user
wgcyeo
69b22e4d206bad330cd7ff8c
Youngwan Lee
admin_assigned
2026-03-12T09:27:29.381Z
641b9dd5f902cc42730f6067
https://cdn-avatars.huggingface.co/v1/production/uploads/641b9dd5f902cc42730f6067/MuDOmsEUHralcoVGmACYG.jpeg
Youngwan Lee
ywlee88
user
ywlee88
69b22e4d206bad330cd7ff8d
Mengye Ren
69b22e4d206bad330cd7ff8e
Sung Ju Hwang
admin_assigned
2026-03-12T09:27:39.671Z
699a783f3fd5ae85cc244693
/avatars/60ef5d9930cc1a273a09c115b0e006f1.svg
Sungju Hwang
sungjuhwang
user
sungjuhwang
MA-EgoQA: Question Answering over Egocentric Videos from Multiple Embodied Agents
As embodied models become powerful, humans will collaborate with multiple embodied AI agents at their workplace or home in the future. To ensure better communication between human users and the multi-agent system, it is crucial to interpret incoming information from agents in parallel and refer to the appropriate context for each query. Existing challenges include effectively compressing and communicating high volumes of individual sensory inputs in the form of video and correctly aggregating multiple egocentric videos to construct system-level memory. In this work, we first formally define a novel problem of understanding multiple long-horizon egocentric videos simultaneously collected from embodied agents. To facilitate research in this direction, we introduce MultiAgent-EgoQA (MA-EgoQA), a benchmark designed to systemically evaluate existing models in our scenario. MA-EgoQA provides 1.7k questions unique to multiple egocentric streams, spanning five categories: social interaction, task coordination, theory-of-mind, temporal reasoning, and environmental interaction. We further propose a simple baseline model for MA-EgoQA named EgoMAS, which leverages shared memory across embodied agents and agent-wise dynamic retrieval. Through comprehensive evaluation across diverse baselines and EgoMAS on MA-EgoQA, we find that current approaches are unable to effectively handle multiple egocentric streams, highlighting the need for future advances in system-level understanding across the agents. The code and benchmark are available at https://ma-egoqa.github.io.
Multi-agent systems require understanding multiple long-horizon egocentric videos simultaneously, necessitating new benchmarks and models for system-level comprehension.
embodied models egocentric videos multi-agent systems system-level memory shared memory dynamic retrieval benchmark MultiAgent-EgoQA EgoMAS