Investigating Anthropometric Fidelity in SAM 3D Body
For researchers in human mesh recovery and medical imaging, this work highlights a critical limitation of current SOTA models in capturing fine-grained anatomical variations, but it is an incremental analysis without empirical results.
The paper identifies that SAM 3D Body fails to reconstruct detailed anthropometric deviations (e.g., geriatric muscle atrophy, scoliosis, pregnancy) due to a 'regression to the mean' effect from its low-dimensional parametric representation and semantic-invariant conditioning. It proposes future pathways like implicit-explicit hybrid representations and Medical-in-the-Loop alignment to extend performance to medical domains.
The release of SAM 3D Body is a recent development in human mesh recovery, demonstrating improved performance in producing clean, topologically coherent meshes from single images. By leveraging the Momentum Human Rig (MHR), it achieves robustness to occlusion and diverse poses. However, our evaluation reveals a specific and consistent limitation: the model struggles to reconstruct detailed anthropometric deviations, particularly in populations exhibiting distinctive morphological alterations such as geriatric muscle atrophy, scoliosis, or pregnancy, even when these features are prominent in the input image. In this paper, we investigate this phenomenon not as a failure of the model's capacity, but as a byproduct of the "perception-distortion trade-off". We posit that the architectural reliance on the low-dimensional parametric MHR representation, combined with semantic-invariant conditioning (DINOv3) and annotation-based alignment, creates a pervasive "regression to the mean" effect. We analyze these mechanisms to understand why individual biological details are smoothed out. Furthermore, we state our contributions by proposing specific, constructive pathways for future work, such as implicit-explicit hybrid representations and Medical-in-the-Loop alignment, to extend the baseline performance of SAM 3D Body into the high-precision medical domain.