SAM-Body4D: Training-Free 4D Human Body Mesh Recovery from Videos
Human Mesh Recovery (HMR) aims to reconstruct 3D human pose and shape from 2D observations and is fundamental to human-centric understanding in real-world scenarios. While recent image-based HMR methods such as SAM 3D Body achieve strong robustness on in-the-wild images, they rely on per-frame inference when applied to videos, leading to temporal inconsistency and degraded performance under occlusions. We address these issues without extra training by leveraging the inherent human continuity in videos. We propose SAM-Body4D, a training-free framework for temporally consistent and occlusion-robust HMR from videos. We first generate identity-consistent masklets using a promptable video segmentation model, then refine them with an Occlusion-Aware module to recover missing regions. The refined masklets guide SAM 3D Body to produce consistent full-body mesh trajectories, while a padding-based parallel strategy enables efficient multi-human inference. Experimental results demonstrate that SAM-Body4D achieves improved temporal stability and robustness in challenging in-the-wild videos, without any retraining. Our code and demo are available at: https://github.com/gaomingqi/sam-body4d.
69395943dfc35938ba12a233
Mingqi Gao
claimed_verified
2025-12-10T15:44:26.374Z
64465292e0dbfe6c21a8af09
https://cdn-avatars.huggingface.co/v1/production/uploads/64465292e0dbfe6c21a8af09/89M79YxG8tJEZz1xH23dI.png
Mingqi Gao
gaomingqi
user
gaomingqi
69395943dfc35938ba12a234
Yunqi Miao
claimed_verified
2025-12-11T10:13:39.955Z
65fad11d88cb799bbe8a18b2
https://cdn-avatars.huggingface.co/v1/production/uploads/65fad11d88cb799bbe8a18b2/CVyVMUjX4nEUBM8ItCHpJ.jpeg
Yunqi Miao
yoqim
user
yoqim
69395943dfc35938ba12a235
Jungong Han
SAM-Body4D: Training-Free 4D Human Body Mesh Recovery from Videos
Human Mesh Recovery (HMR) aims to reconstruct 3D human pose and shape from 2D observations and is fundamental to human-centric understanding in real-world scenarios. While recent image-based HMR methods such as SAM 3D Body achieve strong robustness on in-the-wild images, they rely on per-frame inference when applied to videos, leading to temporal inconsistency and degraded performance under occlusions. We address these issues without extra training by leveraging the inherent human continuity in videos. We propose SAM-Body4D, a training-free framework for temporally consistent and occlusion-robust HMR from videos. We first generate identity-consistent masklets using a promptable video segmentation model, then refine them with an Occlusion-Aware module to recover missing regions. The refined masklets guide SAM 3D Body to produce consistent full-body mesh trajectories, while a padding-based parallel strategy enables efficient multi-human inference. Experimental results demonstrate that SAM-Body4D achieves improved temporal stability and robustness in challenging in-the-wild videos, without any retraining. Our code and demo are available at: https://github.com/gaomingqi/sam-body4d.
SAM-Body4D is a training-free framework that enhances 3D human mesh recovery from videos by ensuring temporal consistency and robustness to occlusions through masklet generation and refinement.
Human Mesh Recovery HMR SAM 3D Body SAM-Body4D video segmentation masklets Occlusion-Aware module full-body mesh trajectories parallel strategy multi-human inference