A Discriminative Vectorial Framework for Multi-modal Feature Representation
This addresses the need for effective multi-modal feature representation in knowledge discovery, particularly for image and multimedia analysis tasks, though it appears incremental as it builds on existing techniques like hashing and correlation analysis.
The paper tackles the problem of extracting useful information from multi-modal data sources by proposing a discriminative vectorial framework that combines multi-modal hashing and discriminative correlation maximization analysis. The framework produces improved results in various applications like image recognition and audio emotion recognition, showing superiority over state-of-the-art statistical machine learning and DNN algorithms.
Due to the rapid advancements of sensory and computing technology, multi-modal data sources that represent the same pattern or phenomenon have attracted growing attention. As a result, finding means to explore useful information from these multi-modal data sources has quickly become a necessity. In this paper, a discriminative vectorial framework is proposed for multi-modal feature representation in knowledge discovery by employing multi-modal hashing (MH) and discriminative correlation maximization (DCM) analysis. Specifically, the proposed framework is capable of minimizing the semantic similarity among different modalities by MH and exacting intrinsic discriminative representations across multiple data sources by DCM analysis jointly, enabling a novel vectorial framework of multi-modal feature representation. Moreover, the proposed feature representation strategy is analyzed and further optimized based on canonical and non-canonical cases, respectively. Consequently, the generated feature representation leads to effective utilization of the input data sources of high quality, producing improved, sometimes quite impressive, results in various applications. The effectiveness and generality of the proposed framework are demonstrated by utilizing classical features and deep neural network (DNN) based features with applications to image and multimedia analysis and recognition tasks, including data visualization, face recognition, object recognition; cross-modal (text-image) recognition and audio emotion recognition. Experimental results show that the proposed solutions are superior to state-of-the-art statistical machine learning (SML) and DNN algorithms.