The Labeled Multiple Canonical Correlation Analysis for Information Fusion
This method addresses the problem of improving pattern recognition accuracy for researchers and practitioners in multimedia processing by providing a generic fusion technique that works with both statistical and deep learning features, though it is incremental as it builds on existing canonical correlation analysis.
The paper tackled multimodal information fusion by introducing Labeled Multiple Canonical Correlation Analysis (LMCCA), which incorporates class labels to create discriminative fused features, resulting in enhanced recognition performance across tasks like handwritten digit, face, object, and emotion recognition.
The objective of multimodal information fusion is to mathematically analyze information carried in different sources and create a new representation which will be more effectively utilized in pattern recognition and other multimedia information processing tasks. In this paper, we introduce a new method for multimodal information fusion and representation based on the Labeled Multiple Canonical Correlation Analysis (LMCCA). By incorporating class label information of the training samples,the proposed LMCCA ensures that the fused features carry discriminative characteristics of the multimodal information representations, and are capable of providing superior recognition performance. We implement a prototype of LMCCA to demonstrate its effectiveness on handwritten digit recognition,face recognition and object recognition utilizing multiple features,bimodal human emotion recognition involving information from both audio and visual domains. The generic nature of LMCCA allows it to take as input features extracted by any means,including those by deep learning (DL) methods. Experimental results show that the proposed method enhanced the performance of both statistical machine learning (SML) methods, and methods based on DL.