Learning Embeddings for Sequential Tasks Using Population of Agents
This work addresses the challenge of task representation in sequential decision-making for reinforcement learning researchers, offering incremental improvements over existing methods.
The paper tackles the problem of learning fixed-dimensional embeddings for tasks in reinforcement learning by using an information-theoretic framework that measures task similarity based on agent performance uncertainty, and it demonstrates effectiveness through quantitative comparisons on applications like predicting agent performance and selecting tasks with desired characteristics.
We present an information-theoretic framework to learn fixed-dimensional embeddings for tasks in reinforcement learning. We leverage the idea that two tasks are similar if observing an agent's performance on one task reduces our uncertainty about its performance on the other. This intuition is captured by our information-theoretic criterion which uses a diverse agent population as an approximation for the space of agents to measure similarity between tasks in sequential decision-making settings. In addition to qualitative assessment, we empirically demonstrate the effectiveness of our techniques based on task embeddings by quantitative comparisons against strong baselines on two application scenarios: predicting an agent's performance on a new task by observing its performance on a small quiz of tasks, and selecting tasks with desired characteristics from a given set of options.