Synthesizing Skeletal Motion and Physiological Signals as a Function of a Virtual Human's Actions and Emotions
This work provides a novel method for generating synthetic human behavioral and physiological data, which is significant for machine learning researchers and healthcare professionals facing data scarcity and privacy concerns in continuous monitoring applications.
This paper introduces a system that synthesizes skeletal motion and physiological signals (ECG, blood pressure, respiration, skin conductance) for a virtual human based on their actions and emotions. The system aims to address the data bottleneck in healthcare ML research by generating high-fidelity, shareable synthetic data, validated through user studies and benchmark comparisons.
Round-the-clock monitoring of human behavior and emotions is required in many healthcare applications which is very expensive but can be automated using machine learning (ML) and sensor technologies. Unfortunately, the lack of infrastructure for collection and sharing of such data is a bottleneck for ML research applied to healthcare. Our goal is to circumvent this bottleneck by simulating a human body in virtual environment. This will allow generation of potentially infinite amounts of shareable data from an individual as a function of his actions, interactions and emotions in a care facility or at home, with no risk of confidentiality breach or privacy invasion. In this paper, we develop for the first time a system consisting of computational models for synchronously synthesizing skeletal motion, electrocardiogram, blood pressure, respiration, and skin conductance signals as a function of an open-ended set of actions and emotions. Our experimental evaluations, involving user studies, benchmark datasets and comparison to findings in the literature, show that our models can generate skeletal motion and physiological signals with high fidelity. The proposed framework is modular and allows the flexibility to experiment with different models. In addition to facilitating ML research for round-the-clock monitoring at a reduced cost, the proposed framework will allow reusability of code and data, and may be used as a training tool for ML practitioners and healthcare professionals.