Self-Augmented Multi-Modal Feature Embedding
This work addresses the challenge of leveraging complementary information from multiple modalities for pattern representation, but it appears incremental as it builds on existing multi-modal and self-augmentation concepts.
The paper tackled the problem of representing patterns across different modalities, such as leaf images and online handwriting, by proposing a self-augmented multi-modal feature embedding method that uses a shared feature space, and demonstrated its effectiveness in classification tasks.
Oftentimes, patterns can be represented through different modalities. For example, leaf data can be in the form of images or contours. Handwritten characters can also be either online or offline. To exploit this fact, we propose the use of self-augmentation and combine it with multi-modal feature embedding. In order to take advantage of the complementary information from the different modalities, the self-augmented multi-modal feature embedding employs a shared feature space. Through experimental results on classification with online handwriting and leaf images, we demonstrate that the proposed method can create effective embeddings.