Analysis of Invariance and Robustness via Invertibility of ReLU-Networks
This work addresses a foundational problem in understanding deep neural network behavior for researchers, though it appears incremental as it builds on existing diagnostic tools without claiming broad breakthroughs.
The paper tackled the lack of a consistent theory on the invertibility of deep neural networks by deriving a theoretical approach to explore preimages of ReLU-layers and mechanisms affecting inverse stability, and numerically demonstrated how this uncovers characteristic network properties.
Studying the invertibility of deep neural networks (DNNs) provides a principled approach to better understand the behavior of these powerful models. Despite being a promising diagnostic tool, a consistent theory on their invertibility is still lacking. We derive a theoretically motivated approach to explore the preimages of ReLU-layers and mechanisms affecting the stability of the inverse. Using the developed theory, we numerically show how this approach uncovers characteristic properties of the network.