DeFM: Learning Foundation Representations from Depth for Robotics
Depth sensors are widely deployed across robotic platforms, and advances in fast, high-fidelity depth simulation have enabled robotic policies trained on depth observations to achieve robust sim-to-real transfer for a wide range of tasks. Despite this, representation learning for depth modality remains underexplored compared to RGB, where large-scale foundation models now define the state of the art. To address this gap, we present DeFM, a self-supervised foundation model trained entirely on depth images for robotic applications. Using a DINO-style self-distillation objective on a curated dataset of 60M depth images, DeFM learns geometric and semantic representations that generalize to diverse environments, tasks, and sensors. To retain metric awareness across multiple scales, we introduce a novel input normalization strategy. We further distill DeFM into compact models suitable for resource-constrained robotic systems. When evaluated on depth-based classification, segmentation, navigation, locomotion, and manipulation benchmarks, DeFM achieves state-of-the-art performance and demonstrates strong generalization from simulation to real-world environments. We release all our pretrained models, which can be adopted off-the-shelf for depth-based robotic learning without task-specific fine-tuning. Webpage: https://de-fm.github.io/
697a1e70df3e800774f13a09
Manthan Patel
claimed_verified
2026-01-29T09:16:58.476Z
66fa8b357aa2514b7fb320f3
/avatars/40e6aee914ce624703d26fc61aafb119.svg
Manthan Patel
manthanhp
user
manthanhp
697a1e70df3e800774f13a0a
Jonas Frey
697a1e70df3e800774f13a0b
Mayank Mittal
697a1e70df3e800774f13a0c
Fan Yang
697a1e70df3e800774f13a0d
Alexander Hansson
697a1e70df3e800774f13a0e
Amir Bar
697a1e70df3e800774f13a0f
Cesar Cadena
697a1e70df3e800774f13a10
Marco Hutter
DeFM: Learning Foundation Representations from Depth for Robotics
Depth sensors are widely deployed across robotic platforms, and advances in fast, high-fidelity depth simulation have enabled robotic policies trained on depth observations to achieve robust sim-to-real transfer for a wide range of tasks. Despite this, representation learning for depth modality remains underexplored compared to RGB, where large-scale foundation models now define the state of the art. To address this gap, we present DeFM, a self-supervised foundation model trained entirely on depth images for robotic applications. Using a DINO-style self-distillation objective on a curated dataset of 60M depth images, DeFM learns geometric and semantic representations that generalize to diverse environments, tasks, and sensors. To retain metric awareness across multiple scales, we introduce a novel input normalization strategy. We further distill DeFM into compact models suitable for resource-constrained robotic systems. When evaluated on depth-based classification, segmentation, navigation, locomotion, and manipulation benchmarks, DeFM achieves state-of-the-art performance and demonstrates strong generalization from simulation to real-world environments. We release all our pretrained models, which can be adopted off-the-shelf for depth-based robotic learning without task-specific fine-tuning. Webpage: https://de-fm.github.io/
DeFM is a self-supervised foundation model for depth representation learning that achieves state-of-the-art performance in robotic tasks through geometric and semantic feature extraction.
self-supervised foundation model DINO-style self-distillation depth images metric awareness input normalization strategy compact models robotic applications geometric representations semantic representations sim-to-real transfer