Cascaded Compressed Sensing Networks: A Reversible Architecture for Layerwise Learning
This addresses a stability and complexity issue in layerwise learning for neural network analysis, but it is incremental as it builds on existing target propagation methods.
The paper tackles the ill-posed problem of target propagation in layerwise learning by modeling each network layer with compressed sensing, eliminating the need for auxiliary networks. Experiments show it achieves better performance than auxiliary network-based methods.
Recently, the method that learns networks layer by layer has attracted increasing interest for its ease of analysis. For the method, the main challenge lies in deriving an optimization target for each layer by inversely propagating the global target of the network. The propagation problem is ill posed, due to involving the inversion of nonlinear activations from lowdimensional to high-dimensional spaces. To address the problem, the existing solution is to learn an auxiliary network to specially propagate the target. However, the network lacks stability, and moreover, it results in higher complexity for network learning. In the letter, we show that target propagation could be achieved by modeling the network s each layer with compressed sensing, without the need of auxiliary networks. Experiments show that the proposed method could achieve better performance than the auxiliary network-based method.