Why every GBDT speed benchmark is wrong
arXiv:1810.10380v1
Originality Synthesis-oriented
AI Analysis
This work addresses benchmarking issues for researchers and practitioners using GBDT algorithms, but it is incremental as it focuses on methodology rather than new algorithms.
The authors identified flaws in existing speed benchmarks for gradient boosted decision trees and proposed requirements for fair and useful benchmarks.
This article provides a comprehensive study of different ways to make speed benchmarks of gradient boosted decision trees algorithm. We show main problems of several straight forward ways to make benchmarks, explain, why a speed benchmarking is a challenging task and provide a set of reasonable requirements for a benchmark to be fair and useful.