Representation Learning for Context-Dependent Decision-Making
This work addresses the challenge of flexible adaptation in changing environments for AI systems, though it appears incremental as it builds on existing representation learning and decision-making frameworks.
The paper tackled the problem of sequential decision-making with contextual changes by proposing an online algorithm for learning and transferring context-dependent representations, which significantly outperformed existing non-adaptive methods. In a case study on the Wisconsin Card Sorting Task, it demonstrated benefits over standard Q-learning and Deep-Q learning algorithms.
Humans are capable of adjusting to changing environments flexibly and quickly. Empirical evidence has revealed that representation learning plays a crucial role in endowing humans with such a capability. Inspired by this observation, we study representation learning in the sequential decision-making scenario with contextual changes. We propose an online algorithm that is able to learn and transfer context-dependent representations and show that it significantly outperforms the existing ones that do not learn representations adaptively. As a case study, we apply our algorithm to the Wisconsin Card Sorting Task, a well-established test for the mental flexibility of humans in sequential decision-making. By comparing our algorithm with the standard Q-learning and Deep-Q learning algorithms, we demonstrate the benefits of adaptive representation learning.