Nonlinear Sufficient Dimension Reduction with a Stochastic Neural Network
This addresses the problem of handling large-scale data in dimension reduction for machine learning practitioners, though it appears incremental as it builds on existing nonlinear methods.
The authors tackled the scalability issue in nonlinear sufficient dimension reduction for large-scale data by proposing a stochastic neural network under a probabilistic framework, showing it compares favorably with state-of-the-art methods and is computationally more efficient.
Sufficient dimension reduction is a powerful tool to extract core information hidden in the high-dimensional data and has potentially many important applications in machine learning tasks. However, the existing nonlinear sufficient dimension reduction methods often lack the scalability necessary for dealing with large-scale data. We propose a new type of stochastic neural network under a rigorous probabilistic framework and show that it can be used for sufficient dimension reduction for large-scale data. The proposed stochastic neural network is trained using an adaptive stochastic gradient Markov chain Monte Carlo algorithm, whose convergence is rigorously studied in the paper as well. Through extensive experiments on real-world classification and regression problems, we show that the proposed method compares favorably with the existing state-of-the-art sufficient dimension reduction methods and is computationally more efficient for large-scale data.