Theoretical guarantees for neural estimators in parametric statistics
This work addresses a foundational gap for researchers in statistics and machine learning, offering a general framework for deriving theoretical guarantees, though it is incremental as it builds on existing empirical observations.
The paper tackles the lack of theoretical guarantees for neural estimators in parametric statistics by analyzing their risk decomposition and providing assumptions for convergence, verified in popular applications.
Neural estimators are simulation-based estimators for the parameters of a family of statistical models, which build a direct mapping from the sample to the parameter vector. They benefit from the versatility of available network architectures and efficient training methods developed in the field of deep learning. Neural estimators are amortized in the sense that, once trained, they can be applied to any new data set with almost no computational cost. While many papers have shown very good performance of these methods in simulation studies and real-world applications, so far no statistical guarantees are available to support these observations theoretically. In this work, we study the risk of neural estimators by decomposing it into several terms that can be analyzed separately. We formulate easy-to-check assumptions ensuring that each term converges to zero, and we verify them for popular applications of neural estimators. Our results provide a general recipe to derive theoretical guarantees also for broader classes of architectures and estimation problems.