Making Learners (More) Monotone
This tackles the issue of unpredictable model performance with increasing data for machine learning practitioners, though it appears incremental as it builds on existing supervised learning methods.
The paper addresses the problem of non-monotonic learning performance, where more data does not always improve model accuracy, by proposing three algorithms to make models more monotone. The algorithm MT_HT achieves less than 1% non-monotone decisions on MNIST while maintaining competitive error rates.
Learning performance can show non-monotonic behavior. That is, more data does not necessarily lead to better models, even on average. We propose three algorithms that take a supervised learning model and make it perform more monotone. We prove consistency and monotonicity with high probability, and evaluate the algorithms on scenarios where non-monotone behaviour occurs. Our proposed algorithm $\text{MT}_{\text{HT}}$ makes less than $1\%$ non-monotone decisions on MNIST while staying competitive in terms of error rate compared to several baselines.