Improving Human Decision-Making by Discovering Efficient Strategies for Hierarchical Planning
This work addresses the challenge of improving human decision-making in complex sequential problems, with potential applications in real-world decision support systems, though it appears incremental as it builds on hierarchical planning concepts.
The paper tackled the problem of discovering efficient planning strategies for human decision-making under computational constraints by introducing a cognitively-inspired reinforcement learning method that decomposes tasks hierarchically, enabling the discovery of strategies that outperform existing algorithms and improve human performance in tasks requiring up to eight steps of planning.
To make good decisions in the real world people need efficient planning strategies because their computational resources are limited. Knowing which planning strategies would work best for people in different situations would be very useful for understanding and improving human decision-making. But our ability to compute those strategies used to be limited to very small and very simple planning tasks. To overcome this computational bottleneck, we introduce a cognitively-inspired reinforcement learning method that can overcome this limitation by exploiting the hierarchical structure of human behavior. The basic idea is to decompose sequential decision problems into two sub-problems: setting a goal and planning how to achieve it. This hierarchical decomposition enables us to discover optimal strategies for human planning in larger and more complex tasks than was previously possible. The discovered strategies outperform existing planning algorithms and achieve a super-human level of computational efficiency. We demonstrate that teaching people to use those strategies significantly improves their performance in sequential decision-making tasks that require planning up to eight steps ahead. By contrast, none of the previous approaches was able to improve human performance on these problems. These findings suggest that our cognitively-informed approach makes it possible to leverage reinforcement learning to improve human decision-making in complex sequential decision-problems. Future work can leverage our method to develop decision support systems that improve human decision making in the real world.