The Artificial Regression Market
This work addresses regression problems for machine learning practitioners by extending an existing technique to a new domain, though it is incremental as it adapts a known method.
The paper generalizes Artificial Prediction Markets from classification to regression by introducing reward kernels and deriving price equations, resulting in two learning rules that outperform Random Forest Regression on many UCI datasets.
The Artificial Prediction Market is a recent machine learning technique for multi-class classification, inspired from the financial markets. It involves a number of trained market participants that bet on the possible outcomes and are rewarded if they predict correctly. This paper generalizes the scope of the Artificial Prediction Markets to regression, where there are uncountably many possible outcomes and the error is usually the MSE. For that, we introduce the reward kernel that rewards each participant based on its prediction error and we derive the price equations. Using two reward kernels we obtain two different learning rules, one of which is approximated using Hermite-Gauss quadrature. The market setting makes it easy to aggregate specialized regressors that only predict when an observation falls into their specialization domain. Experiments show that regression markets based on the two learning rules outperform Random Forest Regression on many UCI datasets and are rarely outperformed.