Learning Situated Awareness in the Real World
A core aspect of human perception is situated awareness, the ability to relate ourselves to the surrounding physical environment and reason over possible actions in context. However, most existing benchmarks for multimodal foundation models (MFMs) emphasize environment-centric spatial relations (relations among objects in a scene), while largely overlooking observer-centric relationships that require reasoning relative to agent's viewpoint, pose, and motion. To bridge this gap, we introduce SAW-Bench (Situated Awareness in the Real World), a novel benchmark for evaluating egocentric situated awareness using real-world videos. SAW-Bench comprises 786 self-recorded videos captured with Ray-Ban Meta (Gen 2) smart glasses spanning diverse indoor and outdoor environments, and over 2,071 human-annotated question-answer pairs. It probes a model's observer-centric understanding with six different awareness tasks. Our comprehensive evaluation reveals a human-model performance gap of 37.66%, even with the best-performing MFM, Gemini 3 Flash. Beyond this gap, our in-depth analysis uncovers several notable findings; for example, while models can exploit partial geometric cues in egocentric videos, they often fail to infer a coherent camera geometry, leading to systematic spatial reasoning errors. We position SAW-Bench as a benchmark for situated spatial intelligence, moving beyond passive observation to understanding physically grounded, observer-centric dynamics.
69967cbb1268a6b79e0d02cf
Chuhan Li
claimed_verified
2026-02-20T08:37:30.993Z
65415f1d5168c4f3487a2103
https://cdn-avatars.huggingface.co/v1/production/uploads/65415f1d5168c4f3487a2103/qJuzDpOGSDL4E1-L2lGWW.jpeg
Chuhan Li
ChuhanLi
user
ChuhanLi
69967cbb1268a6b79e0d02d0
Ruilin Han
69967cbb1268a6b79e0d02d1
Joy Hsu
69967cbb1268a6b79e0d02d2
Yongyuan Liang
69967cbb1268a6b79e0d02d3
Rajiv Dhawan
69967cbb1268a6b79e0d02d4
Jiajun Wu
69967cbb1268a6b79e0d02d5
Ming-Hsuan Yang
69967cbb1268a6b79e0d02d6
Xin Eric Wang
claimed_verified
2026-02-25T17:32:23.240Z
64679a226192d39142245e5e
/avatars/05abee0b6317f100923936ca2099e9eb.svg
Xin Eric Wang
xw-eric
user
xw-eric
Learning Situated Awareness in the Real World
A core aspect of human perception is situated awareness, the ability to relate ourselves to the surrounding physical environment and reason over possible actions in context. However, most existing benchmarks for multimodal foundation models (MFMs) emphasize environment-centric spatial relations (relations among objects in a scene), while largely overlooking observer-centric relationships that require reasoning relative to agent's viewpoint, pose, and motion. To bridge this gap, we introduce SAW-Bench (Situated Awareness in the Real World), a novel benchmark for evaluating egocentric situated awareness using real-world videos. SAW-Bench comprises 786 self-recorded videos captured with Ray-Ban Meta (Gen 2) smart glasses spanning diverse indoor and outdoor environments, and over 2,071 human-annotated question-answer pairs. It probes a model's observer-centric understanding with six different awareness tasks. Our comprehensive evaluation reveals a human-model performance gap of 37.66%, even with the best-performing MFM, Gemini 3 Flash. Beyond this gap, our in-depth analysis uncovers several notable findings; for example, while models can exploit partial geometric cues in egocentric videos, they often fail to infer a coherent camera geometry, leading to systematic spatial reasoning errors. We position SAW-Bench as a benchmark for situated spatial intelligence, moving beyond passive observation to understanding physically grounded, observer-centric dynamics.
SAW-Bench presents a new benchmark for evaluating egocentric situated awareness in multimodal foundation models through real-world video datasets with human-annotated question-answer pairs, focusing on observer-centric spatial reasoning tasks.
multimodal foundation models egocentric videos observer-centric relationships situated awareness spatial reasoning camera geometry real-world videos question-answer pairs