Machine Learning Classifiers Do Not Improve the Prediction of Academic Risk: Evidence from Australia
This work addresses the problem of predicting academic risk in school settings, showing incremental results by challenging the assumption that machine learning always improves prediction over traditional methods.
The study applied popular machine learning models to a large dataset of 1.2 million Australian primary and middle school students to predict academic risk, finding that these models did not outperform logistic regression in identifying students likely to perform below standard on future standardized tests.
Machine learning methods tend to outperform traditional statistical models at prediction. In the prediction of academic achievement, ML models have not shown substantial improvement over logistic regression. So far, these results have almost entirely focused on college achievement, due to the availability of administrative datasets, and have contained relatively small sample sizes by ML standards. In this article we apply popular machine learning models to a large dataset ($n=1.2$ million) containing primary and middle school performance on a standardized test given annually to Australian students. We show that machine learning models do not outperform logistic regression for detecting students who will perform in the `below standard' band of achievement upon sitting their next test, even in a large-$n$ setting.