Unsupervised Deep Learning by Injecting Low-Rank and Sparse Priors
This addresses the challenge of reducing reliance on annotated data for unsupervised learning in computer vision, though it is incremental by applying known priors to existing architectures.
The paper tackles the problem of unsupervised learning in deep neural networks by incorporating sparsity-inducing priors, enabling a U-Net to learn moving objects without annotations and successfully detect foreground objects in test sequences.
What if deep neural networks can learn from sparsity-inducing priors? When the networks are designed by combining layer modules (CNN, RNN, etc), engineers less exploit the inductive bias, i.e., existing well-known rules or prior knowledge, other than annotated training data sets. We focus on employing sparsity-inducing priors in deep learning to encourage the network to concisely capture the nature of high-dimensional data in an unsupervised way. In order to use non-differentiable sparsity-inducing norms as loss functions, we plug their proximal mappings into the automatic differentiation framework. We demonstrate unsupervised learning of U-Net for background subtraction using low-rank and sparse priors. The U-Net can learn moving objects in a training sequence without any annotation, and successfully detect the foreground objects in test sequences.