Why Machine Learning Cannot Ignore Maximum Likelihood Estimation
It targets a foundational problem in machine learning by advocating for more rigorous statistical methods, but it is incremental as it builds on existing theory without new empirical evidence.
The paper argues that machine learning should integrate maximum likelihood estimation for functional parameters to address a lack of statistical rigor and enable inference, but it does not present specific results or concrete numbers.
The growth of machine learning as a field has been accelerating with increasing interest and publications across fields, including statistics, but predominantly in computer science. How can we parse this vast literature for developments that exemplify the necessary rigor? How many of these manuscripts incorporate foundational theory to allow for statistical inference? Which advances have the greatest potential for impact in practice? One could posit many answers to these queries. Here, we assert that one essential idea is for machine learning to integrate maximum likelihood for estimation of functional parameters, such as prediction functions and conditional densities.