ReactDiff: Fundamental Multiple Appropriate Facial Reaction Diffusion Model
This work addresses a critical problem for human-computer interaction systems by improving the realism and appropriateness of facial reactions, though it appears incremental as it builds on existing diffusion models with added constraints.
The paper tackles the challenge of generating diverse and human-like facial reactions in dyadic dialogue by proposing ReactDiff, a temporal diffusion framework that incorporates spatio-temporal facial kinematics and action unit dependencies, achieving state-of-the-art quality, diversity, and appropriateness on the REACT2024 dataset.
The automatic generation of diverse and human-like facial reactions in dyadic dialogue remains a critical challenge for human-computer interaction systems. Existing methods fail to model the stochasticity and dynamics inherent in real human reactions. To address this, we propose ReactDiff, a novel temporal diffusion framework for generating diverse facial reactions that are appropriate for responding to any given dialogue context. Our key insight is that plausible human reactions demonstrate smoothness, and coherence over time, and conform to constraints imposed by human facial anatomy. To achieve this, ReactDiff incorporates two vital priors (spatio-temporal facial kinematics) into the diffusion process: i) temporal facial behavioral kinematics and ii) facial action unit dependencies. These two constraints guide the model toward realistic human reaction manifolds, avoiding visually unrealistic jitters, unstable transitions, unnatural expressions, and other artifacts. Extensive experiments on the REACT2024 dataset demonstrate that our approach not only achieves state-of-the-art reaction quality but also excels in diversity and reaction appropriateness.