Identity Preserve Transform: Understand What Activity Classification Models Have Learnt
This work addresses the issue of model interpretability and generalization failure in activity classification for researchers and practitioners, though it is incremental as it builds on existing methods to analyze model behavior.
The paper tackles the problem of understanding what activity classification models learn by proposing the Identity Preserve Transform (IPT) to manipulate nuisance factors like background and viewpoint while keeping task-related factors unchanged. The result reveals that popular models rely on highly correlated information such as background and objects rather than essential human motion, explaining their poor generalization to unseen datasets.
Activity classification has observed great success recently. The performance on small dataset is almost saturated and people are moving towards larger datasets. What leads to the performance gain on the model and what the model has learnt? In this paper we propose identity preserve transform (IPT) to study this problem. IPT manipulates the nuisance factors (background, viewpoint, etc.) of the data while keeping those factors related to the task (human motion) unchanged. To our surprise, we found popular models are using highly correlated information (background, object) to achieve high classification accuracy, rather than using the essential information (human motion). This can explain why an activity classification model usually fails to generalize to datasets it is not trained on. We implement IPT in two forms, i.e. image-space transform and 3D transform, using synthetic images. The tool will be made open-source to help study model and dataset design.