GAR: An efficient and scalable Graph-based Activity Regularization for semi-supervised learning
It addresses semi-supervised learning for machine learning practitioners, offering a scalable solution, but it appears incremental as it builds on existing graph-based methods.
The paper tackles semi-supervised learning by proposing a graph-based activity regularization method that adapts adjacency using network predictions, resulting in a low-cost framework with performance comparable to state-of-the-art generative approaches.
In this paper, we propose a novel graph-based approach for semi-supervised learning problems, which considers an adaptive adjacency of the examples throughout the unsupervised portion of the training. Adjacency of the examples is inferred using the predictions of a neural network model which is first initialized by a supervised pretraining. These predictions are then updated according to a novel unsupervised objective which regularizes another adjacency, now linking the output nodes. Regularizing the adjacency of the output nodes, inferred from the predictions of the network, creates an easier optimization problem and ultimately provides that the predictions of the network turn into the optimal embedding. Ultimately, the proposed framework provides an effective and scalable graph-based solution which is natural to the operational mechanism of deep neural networks. Our results show comparable performance with state-of-the-art generative approaches for semi-supervised learning on an easier-to-train, low-cost framework.