The return of AdaBoost.MH: multi-class Hamming trees
This work addresses multi-class classification for machine learning practitioners, offering an incremental improvement by enhancing AdaBoost.MH with a novel tree structure.
The authors tackled the multi-class classification problem by proposing vector-valued decision trees within the AdaBoost.MH framework, which avoids reducing it to binary classifications and achieves performance on par with support vector machines and AOSOLogitBoost, while being significantly better than other AdaBoost.MH implementations.
Within the framework of AdaBoost.MH, we propose to train vector-valued decision trees to optimize the multi-class edge without reducing the multi-class problem to $K$ binary one-against-all classifications. The key element of the method is a vector-valued decision stump, factorized into an input-independent vector of length $K$ and label-independent scalar classifier. At inner tree nodes, the label-dependent vector is discarded and the binary classifier can be used for partitioning the input space into two regions. The algorithm retains the conceptual elegance, power, and computational efficiency of binary AdaBoost. In experiments it is on par with support vector machines and with the best existing multi-class boosting algorithm AOSOLogitBoost, and it is significantly better than other known implementations of AdaBoost.MH.