Kazuaki Kondo

Chen Long-fei, Yuichi Nakamura, Kazuaki Kondo

An adaptive guidance system that supports equipment operators requires a comprehensive model, which involves a variety of user behaviors that considers different skill and knowledge levels, as well as rapid-changing task situations. In the present paper, we introduced a novel method for modeling operational tasks, aiming to integrate visual operation records provided by users with diverse experience levels and personal characteristics. For this purpose, we investigated the relationships between user behavior patterns that could be visually observed and their skill levels under machine operation conditions. We considered 144 samples of two sewing tasks performed by 12 operators using a head-mounted RGB-D camera and a static gaze tracker. Behavioral features, such as the operator's gaze and head movements, hand interactions, and hotspots, were observed with significant behavioral trends resulting from continuous user skill improvement. We used a two-step method to model the diversity of user behavior: prototype selection and experience integration based on skill ranking. The experimental results showed that several features could serve as appropriate indices for user skill evaluation, as well as providing valuable clues for revealing personal behavioral characteristics. The integration of user records with different skills and operational habits allowed developing a rich, inclusive task model that could be used flexibly to adapt to diverse user-specific needs.

Kazuaki Kondo, Daisuke Deguchi, Atsushi Shimada

Hand orientation is an essential feature required to understand hand behaviors and subsequently support human activities. In this paper, we present a new method for estimating hand orientation in probability density form. It can solve the cyclicity problem in direct angular representation and enables the integration of multiple predictions based on different features. We validated the performance of the proposed method and an integration example using our dataset, which captured cooperative group work.

Chen Long-fei, Yuichi Nakamura, Kazuaki Kondo

With respect to machine operation tasks, the experiences from different skill level operators, especially novices, can provide worthy understanding about the manner in which they perceive the operational environment and formulate knowledge to deal with various operation situations. In this study, we describe the operator's behaviors by utilizing the relations among their head, hand, and operation location (hotspot) during the operation. A total of 40 experiences associated with a sewing machine operation task performed by amateur operators was recorded via a head-mounted RGB-D camera. We examined important features of operational behaviors in different skill level operators and confirmed their correlation to the difficulties of the operation steps. The result shows that the pure-gazing behavior is significantly reduced when the operator's skill improved. Moreover, the hand-approaching duration and the frequency of attention movement before operation are strongly correlated to the operational difficulty in such machine operating environments.