Multimodal Affect Analysis for Product Feedback Assessment
This addresses product feedback assessment for businesses by analyzing consumer reactions, but it is incremental as it applies existing methods to a specific domain.
The researchers developed a multimodal affect recognition system to classify whether consumers like or dislike products by analyzing facial expressions, body posture, hand gestures, and voice using a Kinect camera and microphone. They achieved real-time performance and evaluated accuracy and feasibility for feedback assessment.
Consumers often react expressively to products such as food samples, perfume, jewelry, sunglasses, and clothing accessories. This research discusses a multimodal affect recognition system developed to classify whether a consumer likes or dislikes a product tested at a counter or kiosk, by analyzing the consumer's facial expression, body posture, hand gestures, and voice after testing the product. A depth-capable camera and microphone system - Kinect for Windows - is utilized. An emotion identification engine has been developed to analyze the images and voice to determine affective state of the customer. The image is segmented using skin color and adaptive threshold. Face, body and hands are detected using the Haar cascade classifier. Canny edges are identified and the lip, body and hand contours are extracted using spatial filtering. Edge count and orientation around the mouth, cheeks, eyes, shoulders, fingers and the location of the edges are used as features. Classification is done by an emotion template mapping algorithm and training a classifier using support vector machines. The real-time performance, accuracy and feasibility for multimodal affect recognition in feedback assessment are evaluated.