Improving approximate RPCA with a k-sparsity prior
This work addresses a specific issue in fast approximate RPCA for representation learning, offering an incremental improvement in performance for classification tasks.
The paper tackled the problem of approximate Robust PCA (RPCA) failing to find parsimonious representations due to bad local minima, and resolved it by using a k-sparsity prior instead of elementwise L1 and L2 priors, resulting in significantly outperforming the original formulation in a discriminative classification task.
A process centric view of robust PCA (RPCA) allows its fast approximate implementation based on a special form o a deep neural network with weights shared across all layers. However, empirically this fast approximation to RPCA fails to find representations that are parsemonious. We resolve these bad local minima by relaxing the elementwise L1 and L2 priors and instead utilize a structure inducing k-sparsity prior. In a discriminative classification task the newly learned representations outperform these from the original approximate RPCA formulation significantly.