Energy-Based Learning for Cooperative Games, with Applications to Valuation Problems in Machine Learning
This work addresses valuation problems for machine learning practitioners, offering a theoretical justification and improved methods, though it is incremental as it builds on existing game-theoretic frameworks.
The authors tackled valuation problems in machine learning, such as feature and data valuation, by proposing a novel energy-based learning approach for cooperative games, which recovers existing game-theoretic criteria and introduces a new Variational Index that shows lower decoupling error and better performance in experiments.
Valuation problems, such as feature interpretation, data valuation and model valuation for ensembles, become increasingly more important in many machine learning applications. Such problems are commonly solved by well-known game-theoretic criteria, such as Shapley value or Banzhaf value. In this work, we present a novel energy-based treatment for cooperative games, with a theoretical justification by the maximum entropy framework. Surprisingly, by conducting variational inference of the energy-based model, we recover various game-theoretic valuation criteria through conducting one-step fixed point iteration for maximizing the mean-field ELBO objective. This observation also verifies the rationality of existing criteria, as they are all attempting to decouple the correlations among the players through the mean-field approach. By running fixed point iteration for multiple steps, we achieve a trajectory of the valuations, among which we define the valuation with the best conceivable decoupling error as the Variational Index. We prove that under uniform initializations, these variational valuations all satisfy a set of game-theoretic axioms. We experimentally demonstrate that the proposed Variational Index enjoys lower decoupling error and better valuation performance on certain synthetic and real-world valuation problems.