Signal Recovery with Non-Expansive Generative Network Priors
This work addresses a theoretical limitation in signal recovery for AI/ML by enabling the use of more realistic, contractive generative networks, which is incremental but broadens applicability in domains like phase retrieval and denoising.
The paper tackles the problem of compressive sensing with deep generative network priors by proving that signals can be recovered from few linear measurements even when the network has contractive layers, relaxing previous expansivity requirements. The result is based on a new matrix concentration inequality (R2WDC) that extends recovery guarantees to non-expansive networks, with layer widths proportional to input size up to log factors.
We study compressive sensing with a deep generative network prior. Initial theoretical guarantees for efficient recovery from compressed linear measurements have been developed for signals in the range of a ReLU network with Gaussian weights and logarithmic expansivity: that is when each layer is larger than the previous one by a logarithmic factor. It was later shown that constant expansivity is sufficient for recovery. It has remained open whether the expansivity can be relaxed, allowing for networks with contractive layers (as often the case of real generators). In this work we answer this question, proving that a signal in the range of a Gaussian generative network can be recovered from few linear measurements provided that the width of the layers is proportional to the input layer size (up to log factors). This condition allows the generative network to have contractive layers. Our result is based on showing that Gaussian matrices satisfy a matrix concentration inequality which we term Range Restricted Weight Distribution Condition (R2WDC) and that weakens the Weight Distribution Condition (WDC) upon which previous theoretical guarantees were based. The WDC has also been used to analyze other signal recovery problems with generative network priors. By replacing the WDC with the R2WDC, we are able to extend previous results for signal recovery with expansive generative network priors to non-expansive ones. We discuss these extensions for phase retrieval, denoising, and spiked matrix recovery.