Adversarial confidence and smoothness regularizations for scalable unsupervised discriminative learning
This addresses the challenge of scalable unsupervised learning for clustering tasks, though it appears incremental as it builds on existing regularization concepts.
The paper tackles the problem of unsupervised discriminative learning by constraining the hypothesis space to piecewise constant functions, resulting in a framework that achieves state-of-the-art clustering results and good generalization on synthetic and real data.
In this paper, we consider a generic probabilistic discriminative learner from the functional viewpoint and argue that, to make it learn well, it is necessary to constrain its hypothesis space to a set of non-trivial piecewise constant functions. To achieve this goal, we present a scalable unsupervised regularization framework. On the theoretical front, we prove that this framework is conducive to a factually confident and smooth discriminative model and connect it to an adversarial Taboo game, spectral clustering and virtual adversarial training. Experimentally, we take deep neural networks as our learners and demonstrate that, when trained under our framework in the unsupervised setting, they not only achieve state-of-the-art clustering results but also generalize well on both synthetic and real data.