Phase Transitions in Image Denoising via Sparsely Coding Convolutional Neural Networks
This provides a method to tune network sparsity for improved denoising performance, but it is incremental as it applies known spin glass techniques to a specific neural network architecture.
The paper tackles the problem of optimizing sparsity in convolutional neural networks for image denoising by analyzing phase transitions, showing that a critical sparsity value minimizes denoising error with power law scaling behavior observed on CIFAR-10 images.
Neural networks are analogous in many ways to spin glasses, systems which are known for their rich set of dynamics and equally complex phase diagrams. We apply well-known techniques in the study of spin glasses to a convolutional sparsely encoding neural network and observe power law finite-size scaling behavior in the sparsity and reconstruction error as the network denoises 32$\times$32 RGB CIFAR-10 images. This finite-size scaling indicates the presence of a continuous phase transition at a critical value of this sparsity. By using the power law scaling relations inherent to finite-size scaling, we can determine the optimal value of sparsity for any network size by tuning the system to the critical point and operate the system at the minimum denoising error.