Primal-dual residual networks
This work addresses the challenge of integrating optimization theory into neural network design for constrained problems, though it appears incremental as it builds on existing primal-dual methods.
The authors tackled the problem of designing deep neural networks inspired by convex optimization, proposing an architecture based on primal-dual splitting methods and showing it outperforms classical splitting methods in speech dequantization.
In this work, we propose a deep neural network architecture motivated by primal-dual splitting methods from convex optimization. We show theoretically that there exists a close relation between the derived architecture and residual networks, and further investigate this connection in numerical experiments. Moreover, we demonstrate how our approach can be used to unroll optimization algorithms for certain problems with hard constraints. Using the example of speech dequantization, we show that our method can outperform classical splitting methods when both are applied to the same task.