Action-Affect Classification and Morphing using Multi-Task Representation Learning
This addresses the problem of recognizing body affect in sequences for applications like human-computer interaction, though it appears incremental as it builds on existing Conditional Restricted Boltzmann Machines.
The paper tackles body affect analysis by proposing a multi-task learning model that learns a shared representation for both action-affect classification and generation, showing superior classification performance over state-of-the-art methods on two public datasets.
Most recent work focused on affect from facial expressions, and not as much on body. This work focuses on body affect analysis. Affect does not occur in isolation. Humans usually couple affect with an action in natural interactions; for example, a person could be talking and smiling. Recognizing body affect in sequences requires efficient algorithms to capture both the micro movements that differentiate between happy and sad and the macro variations between different actions. We depart from traditional approaches for time-series data analytics by proposing a multi-task learning model that learns a shared representation that is well-suited for action-affect classification as well as generation. For this paper we choose Conditional Restricted Boltzmann Machines to be our building block. We propose a new model that enhances the CRBM model with a factored multi-task component to become Multi-Task Conditional Restricted Boltzmann Machines (MTCRBMs). We evaluate our approach on two publicly available datasets, the Body Affect dataset and the Tower Game dataset, and show superior classification performance improvement over the state-of-the-art, as well as the generative abilities of our model.