Machine Learning State-of-the-Art with Uncertainties
This addresses the need for better reporting standards in the ML community, though it is incremental as it builds on existing statistical methods.
The paper tackles the problem of communicating research results in machine learning by demonstrating how confidence intervals around accuracy measurements can enhance clarity and impact reviewing, using an image classification study as an example.
With the availability of data, hardware, software ecosystem and relevant skill sets, the machine learning community is undergoing a rapid development with new architectures and approaches appearing at high frequency every year. In this article, we conduct an exemplary image classification study in order to demonstrate how confidence intervals around accuracy measurements can greatly enhance the communication of research results as well as impact the reviewing process. In addition, we explore the hallmarks and limitations of this approximation. We discuss the relevance of this approach reflecting on a spotlight publication of ICLR22. A reproducible workflow is made available as an open-source adjoint to this publication. Based on our discussion, we make suggestions for improving the authoring and reviewing process of machine learning articles.