Human-Centric Foundation Models: Perception, Generation and Agentic Modeling
It addresses the need for more robust and intelligent digital human modeling, but is incremental as it reviews and categorizes existing approaches rather than introducing new methods.
This survey tackles the problem of unifying diverse human-centric tasks like perception, generation, and agentic modeling into a single framework called Human-centric Foundation Models (HcFMs), aiming to surpass traditional task-specific approaches and provide a roadmap for researchers and practitioners.
Human understanding and generation are critical for modeling digital humans and humanoid embodiments. Recently, Human-centric Foundation Models (HcFMs) inspired by the success of generalist models, such as large language and vision models, have emerged to unify diverse human-centric tasks into a single framework, surpassing traditional task-specific approaches. In this survey, we present a comprehensive overview of HcFMs by proposing a taxonomy that categorizes current approaches into four groups: (1) Human-centric Perception Foundation Models that capture fine-grained features for multi-modal 2D and 3D understanding. (2) Human-centric AIGC Foundation Models that generate high-fidelity, diverse human-related content. (3) Unified Perception and Generation Models that integrate these capabilities to enhance both human understanding and synthesis. (4) Human-centric Agentic Foundation Models that extend beyond perception and generation to learn human-like intelligence and interactive behaviors for humanoid embodied tasks. We review state-of-the-art techniques, discuss emerging challenges and future research directions. This survey aims to serve as a roadmap for researchers and practitioners working towards more robust, versatile, and intelligent digital human and embodiments modeling.