Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training
Humans perceive and understand real-world spaces through a stream of visual observations. Therefore, the ability to streamingly maintain and update spatial evidence from potentially unbounded video streams is essential for spatial intelligence. The core challenge is not simply longer context windows but how spatial information is selected, organized, and retained over time. In this paper, we propose Spatial-TTT towards streaming visual-based spatial intelligence with test-time training (TTT), which adapts a subset of parameters (fast weights) to capture and organize spatial evidence over long-horizon scene videos. Specifically, we design a hybrid architecture and adopt large-chunk updates parallel with sliding-window attention for efficient spatial video processing. To further promote spatial awareness, we introduce a spatial-predictive mechanism applied to TTT layers with 3D spatiotemporal convolution, which encourages the model to capture geometric correspondence and temporal continuity across frames. Beyond architecture design, we construct a dataset with dense 3D spatial descriptions, which guides the model to update its fast weights to memorize and organize global 3D spatial signals in a structured manner. Extensive experiments demonstrate that Spatial-TTT improves long-horizon spatial understanding and achieves state-of-the-art performance on video spatial benchmarks. Project page: https://liuff19.github.io/Spatial-TTT.
69b37d272ad6c5d4cf486aaf
Fangfu Liu
admin_assigned
2026-03-13T08:47:21.963Z
6505a02f9310ce8c400edc63
/avatars/bbf781594fc8c812316711aa8e2797aa.svg
Fangfu Liu
Liuff23
user
Liuff23
69b37d272ad6c5d4cf486ab0
Diankun Wu
admin_assigned
2026-03-13T08:47:31.355Z
6577fba2eb02736add6377f5
/avatars/3e486dd36021feaa4a0259cd89c1eee9.svg
Wu
Diankun
user
Diankun
69b37d272ad6c5d4cf486ab1
Jiawei Chi
admin_assigned
2026-03-13T08:47:43.210Z
66e14cf2fb009ef598305fe5
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/etohfcD9GKaRPir1HRP-5.jpeg
Jiawei Chi
chijw
user
chijw
69b37d272ad6c5d4cf486ab2
Yimo Cai
69b37d272ad6c5d4cf486ab3
Yi-Hsin Hung
69b37d272ad6c5d4cf486ab4
Xumin Yu
admin_assigned
2026-03-13T08:47:57.104Z
6585ad06924e1eafa606912f
https://cdn-avatars.huggingface.co/v1/production/uploads/6585ad06924e1eafa606912f/qHCdQV9_v_iQ2aGOTRSL_.jpeg
XuminYu
XuminYu
user
XuminYu
69b37d272ad6c5d4cf486ab5
Hao Li
69b37d272ad6c5d4cf486ab6
Han Hu
69b37d272ad6c5d4cf486ab7
Yongming Rao
admin_assigned
2026-03-13T08:48:04.416Z
63e4865354f51ea342d45d78
/avatars/2e7eccc878751331ca8b282f53e38899.svg
Yongming Rao
raoyongming
user
raoyongming
69b37d272ad6c5d4cf486ab8
Yueqi Duan
admin_assigned
2026-03-13T08:48:11.717Z
66c8131afafc0fc87ca99650
/avatars/a6eeba2ccf011d5c9964fd38f85bd671.svg
Yueqi Duan
duanyueqi
user
duanyueqi
Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training
Humans perceive and understand real-world spaces through a stream of visual observations. Therefore, the ability to streamingly maintain and update spatial evidence from potentially unbounded video streams is essential for spatial intelligence. The core challenge is not simply longer context windows but how spatial information is selected, organized, and retained over time. In this paper, we propose Spatial-TTT towards streaming visual-based spatial intelligence with test-time training (TTT), which adapts a subset of parameters (fast weights) to capture and organize spatial evidence over long-horizon scene videos. Specifically, we design a hybrid architecture and adopt large-chunk updates parallel with sliding-window attention for efficient spatial video processing. To further promote spatial awareness, we introduce a spatial-predictive mechanism applied to TTT layers with 3D spatiotemporal convolution, which encourages the model to capture geometric correspondence and temporal continuity across frames. Beyond architecture design, we construct a dataset with dense 3D spatial descriptions, which guides the model to update its fast weights to memorize and organize global 3D spatial signals in a structured manner. Extensive experiments demonstrate that Spatial-TTT improves long-horizon spatial understanding and achieves state-of-the-art performance on video spatial benchmarks. Project page: https://liuff19.github.io/Spatial-TTT.
Spatial-TTT enables streaming visual-based spatial intelligence through test-time training that adapts parameters to capture spatial evidence over long video sequences using hybrid architecture and 3D spatiotemporal convolution.
test-time training fast weights spatial evidence long-horizon scene videos hybrid architecture sliding-window attention 3D spatiotemporal convolution spatial-predictive mechanism video spatial benchmarks