GAME: Generalized deep learning model towards multimodal data integration for early screening of adolescent mental disorders
This addresses the global public health challenge of timely identification of mental disorders in adolescents, offering a generalized multimodal screening system with explainable features.
The study tackled the problem of early screening for adolescent mental disorders by developing a multimodal system using a portable robot with an android app, achieving high accuracy (73.34%-92.77%) and F1-Score (71.32%-91.06%) in evaluating mental conditions.
The timely identification of mental disorders in adolescents is a global public health challenge.Single factor is difficult to detect the abnormality due to its complex and subtle nature. Additionally, the generalized multimodal Computer-Aided Screening (CAS) systems with interactive robots for adolescent mental disorders are not available. Here, we design an android application with mini-games and chat recording deployed in a portable robot to screen 3,783 middle school students and construct the multimodal screening dataset, including facial images, physiological signs, voice recordings, and textual transcripts.We develop a model called GAME (Generalized Model with Attention and Multimodal EmbraceNet) with novel attention mechanism that integrates cross-modal features into the model. GAME evaluates adolescent mental conditions with high accuracy (73.34%-92.77%) and F1-Score (71.32%-91.06%).We find each modality contributes dynamically to the mental disorders screening and comorbidities among various mental disorders, indicating the feasibility of explainable model. This study provides a system capable of acquiring multimodal information and constructs a generalized multimodal integration algorithm with novel attention mechanisms for the early screening of adolescent mental disorders.