How Regularization Terms Make Invertible Neural Networks Bayesian Point Estimators
This work provides a theoretical link between regularization and Bayesian estimation for inverse problems, offering a stable and interpretable approach, though it is incremental as it builds on existing invertible network frameworks.
The paper tackles the problem of connecting regularization terms in invertible neural network training to Bayesian point estimators, showing that two specific regularization terms enable the network to approximate the posterior mean and MAP estimator upon inversion.
Can regularization terms in the training of invertible neural networks lead to known Bayesian point estimators in reconstruction? Invertible networks are attractive for inverse problems due to their inherent stability and interpretability. Recently, optimization strategies for invertible neural networks that approximate either a reconstruction map or the forward operator have been studied from a Bayesian perspective, but each has limitations. To address this, we introduce and analyze two regularization terms for the network training that, upon inversion of the network, recover properties of classical Bayesian point estimators: while the first can be connected to the posterior mean, the second resembles the MAP estimator. Our theoretical analysis characterizes how each loss shapes both the learned forward operator and its inverse reconstruction map. Numerical experiments support our findings and demonstrate how these loss-term regularizers introduce data-dependence in a stable and interpretable way.