Nonlinear Multiple Response Regression and Learning of Latent Spaces
This work addresses the need for more interpretable and efficient latent space learning methods in machine learning, though it is incremental as it builds on existing techniques like PCA.
The paper tackles the problem of learning low-dimensional latent spaces from high-dimensional data by introducing a unified method that works in both unsupervised and supervised settings, formulated as a nonlinear multiple-response regression. It demonstrates superior performance in experiments, offering better interpretability and reduced computational complexity compared to traditional methods like autoencoders.
Identifying low-dimensional latent structures within high-dimensional data has long been a central topic in the machine learning community, driven by the need for data compression, storage, transmission, and deeper data understanding. Traditional methods, such as principal component analysis (PCA) and autoencoders (AE), operate in an unsupervised manner, ignoring label information even when it is available. In this work, we introduce a unified method capable of learning latent spaces in both unsupervised and supervised settings. We formulate the problem as a nonlinear multiple-response regression within an index model context. By applying the generalized Stein's lemma, the latent space can be estimated without knowing the nonlinear link functions. Our method can be viewed as a nonlinear generalization of PCA. Moreover, unlike AE and other neural network methods that operate as "black boxes", our approach not only offers better interpretability but also reduces computational complexity while providing strong theoretical guarantees. Comprehensive numerical experiments and real data analyses demonstrate the superior performance of our method.