Synthetically Generating Human-like Data for Sequential Decision Making Tasks via Reward-Shaped Imitation Learning
This addresses the challenge of data scarcity in human-AI systems like games, offering a method to generate realistic synthetic data, though it is incremental as it builds on existing imitation learning and reward shaping techniques.
The paper tackles the problem of generating synthetic human-like data for sequential decision making tasks, proposing a reward-shaped imitation learning algorithm that starts from a small set of human data. The results show that the synthetic data can substitute human data in game-playing tasks with very low divergence, performing almost indistinguishably from humans.
We consider the problem of synthetically generating data that can closely resemble human decisions made in the context of an interactive human-AI system like a computer game. We propose a novel algorithm that can generate synthetic, human-like, decision making data while starting from a very small set of decision making data collected from humans. Our proposed algorithm integrates the concept of reward shaping with an imitation learning algorithm to generate the synthetic data. We have validated our synthetic data generation technique by using the synthetically generated data as a surrogate for human interaction data to solve three sequential decision making tasks of increasing complexity within a small computer game-like setup. Different empirical and statistical analyses of our results show that the synthetically generated data can substitute the human data and perform the game-playing tasks almost indistinguishably, with very low divergence, from a human performing the same tasks.