Functional uniqueness and stability of Gaussian priors in optimal L1 estimation
This work provides a quantitative stability theory for the functional uniqueness of Gaussian priors, which is important for statisticians and machine learning practitioners who use Bayesian estimation.
This paper investigates the relationship between the linearity of optimal Bayesian estimators and the Gaussian nature of prior distributions under Gaussian noise, specifically for L1 and L2 estimation. It establishes that if the optimal estimator is approximately linear, then the prior distribution must be close to Gaussian, providing explicit rates for L2 and a functional-analytic framework for L1.
We study when optimal Bayesian estimators under Gaussian noise are approximately linear, and what this implies about the underlying prior distribution. Consider the classical model \(Y = X + Z\), where \(Z\) is Gaussian and independent of \(X\). It is well known that under squared-error loss, the conditional mean \(\mathbb{E}[X|Y]\) is a linear function of \(Y\) if and only if the prior is Gaussian. Much less is understood under absolute-error loss, where the optimal estimator is the conditional median and standard orthogonality-based tools no longer apply. Recent work has established that, in the Gaussian noise model, the Gaussian prior is also the unique distribution that induces an exactly linear conditional median. In this paper, we move beyond exact characterizations and develop a quantitative stability theory: if the optimal estimator is approximately linear, must the prior be close to Gaussian? For the \(L_2\) setting, we derive explicit rates showing that near-linearity of the conditional mean forces the prior to be close to Gaussian in the Levy metric. For the \(L_1\) setting, we develop a functional-analytic framework based on Hermite expansions and adjoint operators, establishing that approximate linearity of the conditional median implies proximity to the Gaussian family.