TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels
Monocular 3D tracking aims to capture the long-term motion of pixels in 3D space from a single monocular video and has witnessed rapid progress in recent years. However, we argue that the existing monocular 3D tracking methods still fall short in separating the camera motion from foreground dynamic motion and cannot densely track newly emerging dynamic subjects in the videos. To address these two limitations, we propose TrackingWorld, a novel pipeline for dense 3D tracking of almost all pixels within a world-centric 3D coordinate system. First, we introduce a tracking upsampler that efficiently lifts the arbitrary sparse 2D tracks into dense 2D tracks. Then, to generalize the current tracking methods to newly emerging objects, we apply the upsampler to all frames and reduce the redundancy of 2D tracks by eliminating the tracks in overlapped regions. Finally, we present an efficient optimization-based framework to back-project dense 2D tracks into world-centric 3D trajectories by estimating the camera poses and the 3D coordinates of these 2D tracks. Extensive evaluations on both synthetic and real-world datasets demonstrate that our system achieves accurate and dense 3D tracking in a world-centric coordinate frame.
6938e0d4dfc35938ba129f70
Jiahao Lu
6938e0d4dfc35938ba129f71
Weitao Xiong
admin_assigned
2025-12-10T15:50:11.073Z
67bf0cc3dc791d5790d9d6c6
/avatars/c86a991358c3740d99cfa231c9c5a208.svg
Weitao Xiong
xwt123
user
xwt123
6938e0d4dfc35938ba129f72
Jiacheng Deng
6938e0d4dfc35938ba129f73
Peng Li
6938e0d4dfc35938ba129f74
Tianyu Huang
6938e0d4dfc35938ba129f75
Zhiyang Dou
claimed_verified
2026-03-18T08:47:16.244Z
645223fb01d7bd9555ea399a
https://cdn-avatars.huggingface.co/v1/production/uploads/645223fb01d7bd9555ea399a/fVR7XmGg6pMSRKx9sEdvT.png
Zhiyang Dou
frankzydou
user
frankzydou
6938e0d4dfc35938ba129f76
Cheng Lin
6938e0d4dfc35938ba129f77
Sai-Kit Yeung
6938e0d4dfc35938ba129f78
Yuan Liu
TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels
Monocular 3D tracking aims to capture the long-term motion of pixels in 3D space from a single monocular video and has witnessed rapid progress in recent years. However, we argue that the existing monocular 3D tracking methods still fall short in separating the camera motion from foreground dynamic motion and cannot densely track newly emerging dynamic subjects in the videos. To address these two limitations, we propose TrackingWorld, a novel pipeline for dense 3D tracking of almost all pixels within a world-centric 3D coordinate system. First, we introduce a tracking upsampler that efficiently lifts the arbitrary sparse 2D tracks into dense 2D tracks. Then, to generalize the current tracking methods to newly emerging objects, we apply the upsampler to all frames and reduce the redundancy of 2D tracks by eliminating the tracks in overlapped regions. Finally, we present an efficient optimization-based framework to back-project dense 2D tracks into world-centric 3D trajectories by estimating the camera poses and the 3D coordinates of these 2D tracks. Extensive evaluations on both synthetic and real-world datasets demonstrate that our system achieves accurate and dense 3D tracking in a world-centric coordinate frame.
TrackingWorld provides dense 3D tracking of pixels in a world-centric coordinate system by upsampling sparse 2D tracks and optimizing camera poses and 3D coordinates.
monocular 3D tracking tracking upsampler dense 3D tracking world-centric 3D coordinate system 2D tracks camera poses 3D coordinates optimization-based framework synthetic datasets real-world datasets