Joint Dimensionality Reduction for Two Feature Vectors
This addresses multi-modal learning challenges, such as in news classification or cognitive science, by providing a method for efficient feature combination, though it appears incremental as it builds on existing SVD techniques.
The paper tackles the problem of joint dimensionality reduction for two feature vectors in supervised learning, showing that a simple SVD-based algorithm can accurately estimate low-dimensional embeddings with sample complexities that differ only by constant factors across link functions.
Many machine learning problems, especially multi-modal learning problems, have two sets of distinct features (e.g., image and text features in news story classification, or neuroimaging data and neurocognitive data in cognitive science research). This paper addresses the joint dimensionality reduction of two feature vectors in supervised learning problems. In particular, we assume a discriminative model where low-dimensional linear embeddings of the two feature vectors are sufficient statistics for predicting a dependent variable. We show that a simple algorithm involving singular value decomposition can accurately estimate the embeddings provided that certain sample complexities are satisfied, without specifying the nonlinear link function (regressor or classifier). The main results establish sample complexities under multiple settings. Sample complexities for different link functions only differ by constant factors.