Optimal lower Lipschitz bounds for ReLU layers, saturation, and phase retrieval
This work addresses theoretical guarantees for neural network stability and signal recovery, but it appears incremental as it unifies and extends known results.
The paper tackled the problem of deriving lower Lipschitz bounds for ReLU layers and clipping in neural networks, analogous to phase retrieval, and achieved results that are optimal up to a constant factor.
The injectivity of ReLU layers in neural networks, the recovery of vectors from clipped or saturated measurements, and (real) phase retrieval in $\mathbb{R}^n$ allow for a similar problem formulation and characterization using frame theory. In this paper, we revisit all three problems with a unified perspective and derive lower Lipschitz bounds for ReLU layers and clipping which are analogous to the previously known result for phase retrieval and are optimal up to a constant factor.