3AM: Segment Anything with Geometric Consistency in Videos
Video object segmentation methods like SAM2 achieve strong performance through memory-based architectures but struggle under large viewpoint changes due to reliance on appearance features. Traditional 3D instance segmentation methods address viewpoint consistency but require camera poses, depth maps, and expensive preprocessing. We introduce 3AM, a training-time enhancement that integrates 3D-aware features from MUSt3R into SAM2. Our lightweight Feature Merger fuses multi-level MUSt3R features that encode implicit geometric correspondence. Combined with SAM2's appearance features, the model achieves geometry-consistent recognition grounded in both spatial position and visual similarity. We propose a field-of-view aware sampling strategy ensuring frames observe spatially consistent object regions for reliable 3D correspondence learning. Critically, our method requires only RGB input at inference, with no camera poses or preprocessing. On challenging datasets with wide-baseline motion (ScanNet++, Replica), 3AM substantially outperforms SAM2 and extensions, achieving 90.6% IoU and 71.7% Positive IoU on ScanNet++'s Selected Subset, improving over state-of-the-art VOS methods by +15.9 and +30.4 points. Project page: https://jayisaking.github.io/3AM-Page/
6967aa5cc5e371f6b235d2d8
Yang-Che Sun
6967aa5cc5e371f6b235d2d9
Cheng Sun
6967aa5cc5e371f6b235d2da
Chin-Yang Lin
6967aa5cc5e371f6b235d2db
Fu-En Yang
claimed_verified
2026-01-16T10:32:52.998Z
6312cab05beb528b5c1500e3
/avatars/fc27d245dbf06a41c38aa8da880a02cb.svg
Fu-En Yang
FuEnYang
user
FuEnYang
6967aa5cc5e371f6b235d2dc
Min-Hung Chen
claimed_verified
2026-01-15T15:03:57.335Z
64ae22dd1aee69ece065cdcd
https://cdn-avatars.huggingface.co/v1/production/uploads/64ae22dd1aee69ece065cdcd/JG7QaHIrr4i2k4uwR4pZK.png
Min-Hung Chen
cmhungsteve
user
cmhungsteve
6967aa5cc5e371f6b235d2dd
Yen-Yu Lin
6967aa5cc5e371f6b235d2de
Yu-Lun Liu
3AM: Segment Anything with Geometric Consistency in Videos
Video object segmentation methods like SAM2 achieve strong performance through memory-based architectures but struggle under large viewpoint changes due to reliance on appearance features. Traditional 3D instance segmentation methods address viewpoint consistency but require camera poses, depth maps, and expensive preprocessing. We introduce 3AM, a training-time enhancement that integrates 3D-aware features from MUSt3R into SAM2. Our lightweight Feature Merger fuses multi-level MUSt3R features that encode implicit geometric correspondence. Combined with SAM2's appearance features, the model achieves geometry-consistent recognition grounded in both spatial position and visual similarity. We propose a field-of-view aware sampling strategy ensuring frames observe spatially consistent object regions for reliable 3D correspondence learning. Critically, our method requires only RGB input at inference, with no camera poses or preprocessing. On challenging datasets with wide-baseline motion (ScanNet++, Replica), 3AM substantially outperforms SAM2 and extensions, achieving 90.6% IoU and 71.7% Positive IoU on ScanNet++'s Selected Subset, improving over state-of-the-art VOS methods by +15.9 and +30.4 points. Project page: https://jayisaking.github.io/3AM-Page/
3AM enhances video object segmentation by integrating 3D-aware features from MUSt3R into SAM2, achieving improved viewpoint consistency with only RGB input at inference.
SAM2 MUSt3R 3D-aware features Feature Merger multi-level features geometric correspondence spatial position visual similarity field-of-view aware sampling wide-baseline motion IoU Positive IoU