Video-based Social Interaction Behavior Analysis with the Simulated Interaction Task for Children (Kids-SIT)

Accurately quantifying children's social interaction behavior is part of understanding their cognitive and emotional development, as well as mental health conditions. Kids-SIT is a web-based tool designed to computationally analyze children's behaviors by engaging them in a standardized video conversation scenario while their responses are video recorded. In a pre-registered study with 21 healthy children, we evaluated the potential of the Kids-SIT as an accessible paradigm for automated analysis of children's social interaction behavior. We assessed their subjective impression, as well as verbal and non-verbal responses during the Kids-SIT. Verbal content was analyzed using the LIWC tool. Three socially relevant non-verbal behaviors (gaze deviation, smiling, and nodding) were manually annotated and automatically extracted using three computational methods. We examined how well these methods capture naturalistic social interaction patterns of healthy children. We conducted an exploratory classification of healthy children (n=21) and those with social anxiety disorder (n=11) using automated behavioral features. The semantic analysis of the children's verbal responses and their post-hoc impressions indicated that the Kids-SIT successfully elicited natural social interaction behavior. Children's non-verbal behavior also showed similar pattern: they looked at their interaction partner for most of the time, particularly while listening than speaking. Smiling and gazing toward the partner occurred more frequently during the person-directed liked and disliked parts than during the picture-description phase. These non-verbal behavior patterns were captured both by manual annotations and by the computational analysis methods. In the exploratory analysis with a clinical sample, automatically extracted features enabled above-chance differentiation between children with and without SAD (AUC=0.74).