Feature Encodings for Gradient Boosting with Automunge
This work addresses feature encoding selection for gradient boosting in tabular data preprocessing, providing empirical validation for defaults in the Automunge library, which is incremental as it benchmarks existing methods.
The study validated Automunge's default feature encodings for gradient boosting by benchmarking on diverse datasets, finding that binarization for categoric features and z-score normalization for numeric features were top performers in tuning duration and model performance, with one-hot encoding underperforming compared to binarization.
Automunge is a tabular preprocessing library that encodes dataframes for supervised learning. When selecting a default feature encoding strategy for gradient boosted learning, one may consider metrics of training duration and achieved predictive performance associated with the feature representations. Automunge offers a default of binarization for categoric features and z-score normalization for numeric. The presented study sought to validate those defaults by way of benchmarking on a series of diverse data sets by encoding variations with tuned gradient boosted learning. We found that on average our chosen defaults were top performers both from a tuning duration and a model performance standpoint. Another key finding was that one hot encoding did not perform in a manner consistent with suitability to serve as a categoric default in comparison to categoric binarization. We present here these and further benchmarks.