A Comparative Analysis of XGBoost
This work provides practical insights for machine learning practitioners on when to use XGBoost, but it is incremental as it builds on existing methods without introducing new techniques.
This paper conducted a comparative analysis of XGBoost against random forests and gradient boosting, evaluating training speed, generalization performance, and parameter tuning, and found that XGBoost is not always the best choice in all scenarios.
XGBoost is a scalable ensemble technique based on gradient boosting that has demonstrated to be a reliable and efficient machine learning challenge solver. This work proposes a practical analysis of how this novel technique works in terms of training speed, generalization performance and parameter setup. In addition, a comprehensive comparison between XGBoost, random forests and gradient boosting has been performed using carefully tuned models as well as using the default settings. The results of this comparison may indicate that XGBoost is not necessarily the best choice under all circumstances. Finally an extensive analysis of XGBoost parametrization tuning process is carried out.