Mean-Squared Accuracy of Good-Turing Estimator
This work provides theoretical bounds for a widely used estimator in statistics and machine learning, but it is incremental as it focuses on error characterization rather than introducing a new method.
The paper characterizes the maximal mean-squared error of the Good-Turing estimator for any sample and alphabet size, addressing the problem of estimating objects not occurring in a sample, with applications in language modeling and ecology.
The brilliant method due to Good and Turing allows for estimating objects not occurring in a sample. The problem, known under names "sample coverage" or "missing mass" goes back to their cryptographic work during WWII, but over years has found has many applications, including language modeling, inference in ecology and estimation of distribution properties. This work characterizes the maximal mean-squared error of the Good-Turing estimator, for any sample \emph{and} alphabet size.