Transformer-based Parameter Estimation in Statistics
This addresses the problem of simplifying and potentially improving parameter estimation for statisticians and data scientists, though it appears incremental as it applies an existing AI method to a traditional statistical task.
The paper tackles parameter estimation in statistics by proposing a transformer-based approach that eliminates the need for closed-form solutions, mathematical derivations, or knowledge of probability density functions, achieving similar or better accuracy than maximum likelihood estimation on distributions like normal, exponential, and beta as measured by mean-square-errors.
Parameter estimation is one of the most important tasks in statistics, and is key to helping people understand the distribution behind a sample of observations. Traditionally parameter estimation is done either by closed-form solutions (e.g., maximum likelihood estimation for Gaussian distribution), or by iterative numerical methods such as Newton-Raphson method when closed-form solution does not exist (e.g., for Beta distribution). In this paper we propose a transformer-based approach to parameter estimation. Compared with existing solutions, our approach does not require a closed-form solution or any mathematical derivations. It does not even require knowing the probability density function, which is needed by numerical methods. After the transformer model is trained, only a single inference is needed to estimate the parameters of the underlying distribution based on a sample of observations. In the empirical study we compared our approach with maximum likelihood estimation on commonly used distributions such as normal distribution, exponential distribution and beta distribution. It is shown that our approach achieves similar or better accuracy as measured by mean-square-errors.