Action Assembly: Sparse Imitation Learning for Text Based Games with Combinatorial Action Spaces
This addresses the challenge of scaling imitation learning to complex environments with vast action spaces, which is incremental as it builds on existing compressed sensing and imitation learning methods.
The authors tackled the problem of solving text-based games with large combinatorial action spaces by proposing a computationally efficient algorithm that combines compressed sensing with imitation learning, achieving success in solving the entire game of Zork1 with an action space of about 10 million actions using both perfect and noisy demonstrations.
We propose a computationally efficient algorithm that combines compressed sensing with imitation learning to solve text-based games with combinatorial action spaces. Specifically, we introduce a new compressed sensing algorithm, named IK-OMP, which can be seen as an extension to the Orthogonal Matching Pursuit (OMP). We incorporate IK-OMP into a supervised imitation learning setting and show that the combined approach (Sparse Imitation Learning, Sparse-IL) solves the entire text-based game of Zork1 with an action space of approximately 10 million actions given both perfect and noisy demonstrations.