Learning Deep Representations By Distributed Random Samplings
This addresses the problem of efficient unsupervised learning for dimensionality reduction, though it appears incremental as it builds on existing clustering and sampling ideas.
The paper tackles unsupervised nonlinear dimensionality reduction by proposing a deep model based on distributed random samplings, which learns abstract representations and is much faster than deep neural networks on large-scale problems.
In this paper, we propose an extremely simple deep model for the unsupervised nonlinear dimensionality reduction -- deep distributed random samplings, which performs like a stack of unsupervised bootstrap aggregating. First, its network structure is novel: each layer of the network is a group of mutually independent $k$-centers clusterings. Second, its learning method is extremely simple: the $k$ centers of each clustering are only $k$ randomly selected examples from the training data; for small-scale data sets, the $k$ centers are further randomly reconstructed by a simple cyclic-shift operation. Experimental results on nonlinear dimensionality reduction show that the proposed method can learn abstract representations on both large-scale and small-scale problems, and meanwhile is much faster than deep neural networks on large-scale problems.