Φeat: Physically-Grounded Feature Representation
Foundation models have emerged as effective backbones for many vision tasks. However, current self-supervised features entangle high-level semantics with low-level physical factors, such as geometry and illumination, hindering their use in tasks requiring explicit physical reasoning. In this paper, we introduce Φeat, a novel physically-grounded visual backbone that encourages a representation sensitive to material identity, including reflectance cues and geometric mesostructure. Our key idea is to employ a pretraining strategy that contrasts spatial crops and physical augmentations of the same material under varying shapes and lighting conditions. While similar data have been used in high-end supervised tasks such as intrinsic decomposition or material estimation, we demonstrate that a pure self-supervised training strategy, without explicit labels, already provides a strong prior for tasks requiring robust features invariant to external physical factors. We evaluate the learned representations through feature similarity analysis and material selection, showing that Φeat captures physically-grounded structure beyond semantic grouping. These findings highlight the promise of unsupervised physical feature learning as a foundation for physics-aware perception in vision and graphics. These findings highlight the promise of unsupervised physical feature learning as a foundation for physics-aware perception in vision and graphics.
691d7cd30247f3d258ef3311
Giuseppe Vecchio
claimed_verified
2025-11-19T08:38:21.153Z
643fa1c718afbc4d1f3e5e94
https://cdn-avatars.huggingface.co/v1/production/uploads/643fa1c718afbc4d1f3e5e94/_cg3HHuycPGHnexJHbfwP.jpeg
Giuseppe Vecchio
gvecchio
user
gvecchio
691d7cd30247f3d258ef3312
Adrien Kaiser
691d7cd30247f3d258ef3313
Rouffet Romain
691d7cd30247f3d258ef3314
Rosalie Martin
691d7cd30247f3d258ef3315
Elena Garces
691d7cd30247f3d258ef3316
Tamy Boubekeur
Φeat: Physically-Grounded Feature Representation
Foundation models have emerged as effective backbones for many vision tasks. However, current self-supervised features entangle high-level semantics with low-level physical factors, such as geometry and illumination, hindering their use in tasks requiring explicit physical reasoning. In this paper, we introduce Φeat, a novel physically-grounded visual backbone that encourages a representation sensitive to material identity, including reflectance cues and geometric mesostructure. Our key idea is to employ a pretraining strategy that contrasts spatial crops and physical augmentations of the same material under varying shapes and lighting conditions. While similar data have been used in high-end supervised tasks such as intrinsic decomposition or material estimation, we demonstrate that a pure self-supervised training strategy, without explicit labels, already provides a strong prior for tasks requiring robust features invariant to external physical factors. We evaluate the learned representations through feature similarity analysis and material selection, showing that Φeat captures physically-grounded structure beyond semantic grouping. These findings highlight the promise of unsupervised physical feature learning as a foundation for physics-aware perception in vision and graphics. These findings highlight the promise of unsupervised physical feature learning as a foundation for physics-aware perception in vision and graphics.
A physically-grounded visual backbone, $Φ$eat, is introduced to capture material identity through self-supervised training, demonstrating robust features invariant to external physical factors.
physically-grounded visual backbone self-supervised training spatial crops physical augmentations material identity reflectance cues geometric mesostructure intrinsic decomposition material estimation physics-aware perception