Signal Recovery from Pooling Representations
This work addresses the challenge of signal recovery in neural networks for researchers in machine learning, but it is incremental as it builds on existing phase recovery methods.
The paper tackled the problem of inverting pooling layers in neural networks by computing lower Lipschitz bounds for various pooling operators, and it demonstrated that pooling layers can be inverted using phase recovery algorithms, with empirical verification on MNIST and image patches.
In this work we compute lower Lipschitz bounds of $\ell_p$ pooling operators for $p=1, 2, \infty$ as well as $\ell_p$ pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm that pooling layers can be inverted with phase recovery algorithms. Moreover, the regularity of the inverse pooling, controlled by the lower Lipschitz constant, is empirically verified with a nearest neighbor regression.