Cambrian-S: Towards Spatial Supersensing in Video
We argue that progress in true multimodal intelligence calls for a shift from
reactive, task-driven systems and brute-force long context towards a broader
paradigm of supersensing. We frame spatial supersensing as four stages beyond
linguistic-only understanding: semantic perception (naming what is seen),
streaming event cognition (maintaining memory across continuous experiences),
implicit 3D spatial cognition (inferring the world behind pixels), and
predictive world modeling (creating internal models that filter and organize
information). Current benchmarks largely test only the early stages, offering
narrow coverage of spatial cognition and rarely challenging models in ways that
require true world modeling. To drive progress in spatial supersensing, we
present VSI-SUPER, a two-part benchmark: VSR (long-horizon visual spatial
recall) and VSC (continual visual spatial counting). These tasks require
arbitrarily long video inputs yet are resistant to brute-force context
expansion. We then test data scaling limits by curating VSI-590K and training
Cambrian-S, achieving +30% absolute improvement on VSI-Bench without
sacrificing general capabilities. Yet performance on VSI-SUPER remains limited,
indicating that scale alone is insufficient for spatial supersensing. We
propose predictive sensing as a path forward, presenting a proof-of-concept in
which a self-supervised next-latent-frame predictor leverages surprise
(prediction error) to drive memory and event segmentation. On VSI-SUPER, this
approach substantially outperforms leading proprietary baselines, showing that
spatial supersensing requires models that not only see but also anticipate,
select, and organize experience.
690d5b7aad2597bf6c464cb9
Shusheng Yang
claimed_verified
2025-11-07T10:28:45.693Z
627ccf058b4e56cfc2716425
https://cdn-avatars.huggingface.co/v1/production/uploads/1652346592327-noauth.jpeg
Shusheng Yang
ShushengYang
user
ShushengYang
690d5b7aad2597bf6c464cba
Jihan Yang
claimed_verified
2025-11-17T10:32:11.880Z
6304baf041387c7f1177a5d2
https://cdn-avatars.huggingface.co/v1/production/uploads/6304baf041387c7f1177a5d2/cQgCR8AsrMUaF2QVh97I9.jpeg
Jihan Yang
jihanyang
user
jihanyang
690d5b7aad2597bf6c464cbb
Pinzhi Huang
claimed_verified
2025-11-10T09:31:35.377Z
65d14d80818f0593463fee32
https://cdn-avatars.huggingface.co/v1/production/uploads/65d14d80818f0593463fee32/5dG3GwfzuMA9j_DhNMVsg.jpeg
Pinzhi Huang
EdwinHuang
user
EdwinHuang
690d5b7aad2597bf6c464cbc
Ellis Brown
claimed_verified
2025-11-10T09:31:37.284Z
626dc5105f7327906f0b2a4e
https://cdn-avatars.huggingface.co/v1/production/uploads/626dc5105f7327906f0b2a4e/QCSzuwYqsv8ozRnusVb-F.jpeg
Ellis Brown
ellisbrown
user
ellisbrown
690d5b7aad2597bf6c464cbd
Zihao Yang
690d5b7aad2597bf6c464cbe
Yue Yu
690d5b7aad2597bf6c464cbf
Shengbang Tong
690d5b7aad2597bf6c464cc0
Zihan Zheng
690d5b7aad2597bf6c464cc1
Yifan Xu
690d5b7aad2597bf6c464cc2
Muhan Wang
690d5b7aad2597bf6c464cc3
Daohan Lu
claimed_verified
2026-03-05T09:58:20.903Z
634eb33baae4bde2c8df5660
/avatars/327281827cc7632f5f5615cffa7abe7d.svg
Daohan Lu
daohanlu
user
daohanlu
690d5b7aad2597bf6c464cc4
Rob Fergus
690d5b7aad2597bf6c464cc5
Yann LeCun
690d5b7aad2597bf6c464cc6
Li Fei-Fei
690d5b7aad2597bf6c464cc7
Saining Xie
claimed_verified
2025-11-11T19:51:34.053Z
6596422646624a86ff3b3bda
/avatars/216e12b77e45ac5f1fa20932f5745411.svg
Saining Xie
sainx
user
sainx
Cambrian-S: Towards Spatial Supersensing in Video
We argue that progress in true multimodal intelligence calls for a shift from
reactive, task-driven systems and brute-force long context towards a broader
paradigm of supersensing. We frame spatial supersensing as four stages beyond
linguistic-only understanding: semantic perception (naming what is seen),
streaming event cognition (maintaining memory across continuous experiences),
implicit 3D spatial cognition (inferring the world behind pixels), and
predictive world modeling (creating internal models that filter and organize
information). Current benchmarks largely test only the early stages, offering
narrow coverage of spatial cognition and rarely challenging models in ways that
require true world modeling. To drive progress in spatial supersensing, we
present VSI-SUPER, a two-part benchmark: VSR (long-horizon visual spatial
recall) and VSC (continual visual spatial counting). These tasks require
arbitrarily long video inputs yet are resistant to brute-force context
expansion. We then test data scaling limits by curating VSI-590K and training
Cambrian-S, achieving +30% absolute improvement on VSI-Bench without
sacrificing general capabilities. Yet performance on VSI-SUPER remains limited,
indicating that scale alone is insufficient for spatial supersensing. We
propose predictive sensing as a path forward, presenting a proof-of-concept in
which a self-supervised next-latent-frame predictor leverages surprise
(prediction error) to drive memory and event segmentation. On VSI-SUPER, this
approach substantially outperforms leading proprietary baselines, showing that
spatial supersensing requires models that not only see but also anticipate,
select, and organize experience.
Progress in multimodal intelligence requires a shift to supersensing, including semantic perception, event cognition, spatial cognition, and predictive modeling, demonstrated through VSI-SUPER benchmarks and a self-supervised predictive sensing approach.
supersensing semantic perception streaming event cognition implicit 3D spatial cognition predictive world modeling VSI-SUPER VSR VSC VSI-590K Cambrian-S VSI-Bench predictive sensing self-supervised next-latent-frame predictor surprise prediction error memory event segmentation