Learning to Optimize via Wasserstein Deep Inverse Optimal Control
This work addresses the problem of modeling user behavior in social sciences for applications like recommender systems, though it appears incremental as it builds upon existing inverse optimal control and generative adversarial network techniques.
The paper tackles the inverse optimal control problem in social sciences by learning a user's cost function from observed behavior, proposing a novel variational principle that treats the user as a reinforcement learning agent. It introduces a two-step Wasserstein framework and demonstrates significant performance gains in recommender systems and social networks compared to existing methods.
We study the inverse optimal control problem in social sciences: we aim at learning a user's true cost function from the observed temporal behavior. In contrast to traditional phenomenological works that aim to learn a generative model to fit the behavioral data, we propose a novel variational principle and treat user as a reinforcement learning algorithm, which acts by optimizing his cost function. We first propose a unified KL framework that generalizes existing maximum entropy inverse optimal control methods. We further propose a two-step Wasserstein inverse optimal control framework. In the first step, we compute the optimal measure with a novel mass transport equation. In the second step, we formulate the learning problem as a generative adversarial network. In two real world experiments - recommender systems and social networks, we show that our framework obtains significant performance gains over both existing inverse optimal control methods and point process based generative models.