BET: Bayesian Ensemble Trees for Clustering and Prediction in Heterogeneous Data
This work addresses data heterogeneity in machine learning applications, such as medical classification and regression, with an incremental method combining existing techniques.
The paper tackles the problem of clustering and prediction in heterogeneous data by proposing Bayesian Ensemble Trees (BET), a tree-averaging model that uses a Dirichlet process mixture to group data subsets into ensembles, resulting in improved prediction performance with fewer trees compared to bootstrap-aggregating.
We propose a novel "tree-averaging" model that utilizes the ensemble of classification and regression trees (CART). Each constituent tree is estimated with a subset of similar data. We treat this grouping of subsets as Bayesian ensemble trees (BET) and model them as an infinite mixture Dirichlet process. We show that BET adapts to data heterogeneity and accurately estimates each component. Compared with the bootstrap-aggregating approach, BET shows improved prediction performance with fewer trees. We develop an efficient estimating procedure with improved sampling strategies in both CART and mixture models. We demonstrate these advantages of BET with simulations, classification of breast cancer and regression of lung function measurement of cystic fibrosis patients. Keywords: Bayesian CART; Dirichlet Process; Ensemble Approach; Heterogeneity; Mixture of Trees.