Equivariant Deep Equilibrium Models for Imaging Inverse Problems
This work addresses a specific training bottleneck for researchers in computational imaging, offering an incremental improvement in method implementation.
The paper tackled the challenge of training deep equilibrium models (DEQs) for imaging inverse problems using equivariant imaging, which avoids ground truth data by leveraging signal symmetries, and showed that modular backpropagation simplifies training, with experiments demonstrating DEQs trained with implicit differentiation outperform baseline methods.
Equivariant imaging (EI) enables training signal reconstruction models without requiring ground truth data by leveraging signal symmetries. Deep equilibrium models (DEQs) are a powerful class of neural networks where the output is a fixed point of a learned operator. However, training DEQs with complex EI losses requires implicit differentiation through fixed-point computations, whose implementation can be challenging. We show that backpropagation can be implemented modularly, simplifying training. Experiments demonstrate that DEQs trained with implicit differentiation outperform those trained with Jacobian-free backpropagation and other baseline methods. Additionally, we find evidence that EI-trained DEQs approximate the proximal map of an invariant prior.