Not quite unreasonable effectiveness of machine learning algorithms
This addresses the problem of evaluating machine learning models for researchers and practitioners, but it is incremental as it critiques existing metrics without proposing a concrete new solution.
The paper argues that the near-perfect performance of state-of-the-art machine learning algorithms may stem from low sample variability and effective learning of typical patterns, suggesting that standard metrics fail to reveal model capacity and new metrics are needed for better understanding.
State-of-the-art machine learning algorithms demonstrate close to absolute performance in selected challenges. We provide arguments that the reason can be in low variability of the samples and high effectiveness in learning typical patterns. Due to this fact, standard performance metrics do not reveal model capacity and new metrics are required for the better understanding of state-of-the-art.