Building Usage Profiles Using Deep Neural Nets
This incremental work addresses the challenge of profiling software usage for large, diverse customer bases, potentially aiding software engineering activities like testing.
The authors tackled the problem of building software usage profiles from instructional videos by using a Deep Convolutional Neural Network to recognize user actions, achieving a mean average precision of 94.42% in classifying five actions across 236 Microsoft Word tutorial videos.
To improve software quality, one needs to build test scenarios resembling the usage of a software product in the field. This task is rendered challenging when a product's customer base is large and diverse. In this scenario, existing profiling approaches, such as operational profiling, are difficult to apply. In this work, we consider publicly available video tutorials of a product to profile usage. Our goal is to construct an automatic approach to extract information about user actions from instructional videos. To achieve this goal, we use a Deep Convolutional Neural Network (DCNN) to recognize user actions. Our pilot study shows that a DCNN trained to recognize user actions in video can classify five different actions in a collection of 236 publicly available Microsoft Word tutorial videos (published on YouTube). In our empirical evaluation we report a mean average precision of 94.42% across all actions. This study demonstrates the efficacy of DCNN-based methods for extracting software usage information from videos. Moreover, this approach may aid in other software engineering activities that require information about customer usage of a product.