Learning with Subset Stacking
This addresses regression problems where relationships vary across the predictor space, but it appears incremental as it builds on stacking methods.
The authors tackled regression with heterogeneous input-output relations by proposing LESS, a new algorithm that generates subsets, trains local predictors, and combines them in a novel way, showing it is highly competitive against state-of-the-art methods on several datasets.
We propose a new regression algorithm that learns from a set of input-output pairs. Our algorithm is designed for populations where the relation between the input variables and the output variable exhibits a heterogeneous behavior across the predictor space. The algorithm starts with generating subsets that are concentrated around random points in the input space. This is followed by training a local predictor for each subset. Those predictors are then combined in a novel way to yield an overall predictor. We call this algorithm "LEarning with Subset Stacking" or LESS, due to its resemblance to the method of stacking regressors. We offer bagging and boosting variants of LESS and test against the state-of-the-art methods on several datasets. Our comparison shows that LESS is highly competitive.