A Learning Algorithm That Attains the Human Optimum in a Repeated Human-Machine Interaction Game
This addresses the challenge of designing adaptive control systems, such as exoskeletons, for human users by providing a more robust method to optimize performance without needing to infer cost functions directly.
The paper tackles the problem of minimizing a human's unknown cost function in human-machine interactions by introducing a game-theoretic learning algorithm that avoids solving ill-posed inverse problems, and demonstrates through human experiments that it consistently converges to the cost minimum in both scalar and multidimensional scenarios.
When humans interact with learning-based control systems, a common goal is to minimize a cost function known only to the human. For instance, an exoskeleton may adapt its assistance in an effort to minimize the human's metabolic cost-of-transport. Conventional approaches to synthesizing the learning algorithm solve an inverse problem to infer the human's cost. However, these problems can be ill-posed, hard to solve, or sensitive to problem data. Here we show a game-theoretic learning algorithm that works solely by observing human actions to find the cost minimum, avoiding the need to solve an inverse problem. We evaluate the performance of our algorithm in an extensive set of human subjects experiments, demonstrating consistent convergence to the minimum of a prescribed human cost function in scalar and multidimensional instantiations of the game. We conclude by outlining future directions for theoretical and empirical extensions of our results.