Feature Selection by a Mechanism Design
This addresses feature selection for econometric and statistical modeling, offering a novel game-theoretic method that is incremental in improving accuracy over existing techniques.
The paper tackles feature selection in econometric models by framing it as a coalitional game where irrelevant features are dummy players, using a mechanism design to match model performance with marginal effects and dropping features with insignificant valuations. In simulations, the new approach significantly outperforms popular methods, with robust accuracy across payoff functions.
In constructing an econometric or statistical model, we pick relevant features or variables from many candidates. A coalitional game is set up to study the selection problem where the players are the candidates and the payoff function is a performance measurement in all possible modeling scenarios. Thus, in theory, an irrelevant feature is equivalent to a dummy player in the game, which contributes nothing to all modeling situations. The hypothesis test of zero mean contribution is the rule to decide a feature is irrelevant or not. In our mechanism design, the end goal perfectly matches the expected model performance with the expected sum of individual marginal effects. Within a class of noninformative likelihood among all modeling opportunities, the matching equation results in a specific valuation for each feature. After estimating the valuation and its standard deviation, we drop any candidate feature if its valuation is not significantly different from zero. In the simulation studies, our new approach significantly outperforms several popular methods used in practice, and its accuracy is robust to the choice of the payoff function.