VideoMaMa: Mask-Guided Video Matting via Generative Prior
Generalizing video matting models to real-world videos remains a significant challenge due to the scarcity of labeled data. To address this, we present Video Mask-to-Matte Model (VideoMaMa) that converts coarse segmentation masks into pixel accurate alpha mattes, by leveraging pretrained video diffusion models. VideoMaMa demonstrates strong zero-shot generalization to real-world footage, even though it is trained solely on synthetic data. Building on this capability, we develop a scalable pseudo-labeling pipeline for large-scale video matting and construct the Matting Anything in Video (MA-V) dataset, which offers high-quality matting annotations for more than 50K real-world videos spanning diverse scenes and motions. To validate the effectiveness of this dataset, we fine-tune the SAM2 model on MA-V to obtain SAM2-Matte, which outperforms the same model trained on existing matting datasets in terms of robustness on in-the-wild videos. These findings emphasize the importance of large-scale pseudo-labeled video matting and showcase how generative priors and accessible segmentation cues can drive scalable progress in video matting research.
697324effb12c92b735b7592
Sangbeom Lim
claimed_verified
2026-01-23T09:38:12.611Z
64842d1edc475c315e41123a
https://cdn-avatars.huggingface.co/v1/production/uploads/64842d1edc475c315e41123a/Yqam_dy0MnQm4Rn1_1_4N.jpeg
Sangbeom Lim
SammyLim
user
SammyLim
697324effb12c92b735b7593
Seoung Wug Oh
697324effb12c92b735b7594
Jiahui Huang
697324effb12c92b735b7595
Heeji Yoon
697324effb12c92b735b7596
Seungryong Kim
697324effb12c92b735b7597
Joon-Young Lee
VideoMaMa: Mask-Guided Video Matting via Generative Prior
Generalizing video matting models to real-world videos remains a significant challenge due to the scarcity of labeled data. To address this, we present Video Mask-to-Matte Model (VideoMaMa) that converts coarse segmentation masks into pixel accurate alpha mattes, by leveraging pretrained video diffusion models. VideoMaMa demonstrates strong zero-shot generalization to real-world footage, even though it is trained solely on synthetic data. Building on this capability, we develop a scalable pseudo-labeling pipeline for large-scale video matting and construct the Matting Anything in Video (MA-V) dataset, which offers high-quality matting annotations for more than 50K real-world videos spanning diverse scenes and motions. To validate the effectiveness of this dataset, we fine-tune the SAM2 model on MA-V to obtain SAM2-Matte, which outperforms the same model trained on existing matting datasets in terms of robustness on in-the-wild videos. These findings emphasize the importance of large-scale pseudo-labeled video matting and showcase how generative priors and accessible segmentation cues can drive scalable progress in video matting research.
VideoMaMa converts coarse masks to accurate alpha mattes using pretrained video diffusion models, enabling zero-shot generalization and scalable pseudo-labeling for video matting.
video matting video diffusion models pseudo-labeling Matting Anything in Video SAM2 alpha mattes coarse segmentation masks zero-shot generalization