Improving Human Decision-making by Discovering Efficient Strategies for Hierarchical Planning

Abstract

To make good decisions in the real world, people need efficient planning strategies because their computational resources are limited. Knowing which planning strategies would work best for people in different situations would be very useful for understanding and improving human decision-making. Our ability to compute those strategies used to be limited to very small and very simple planning tasks. Here, we introduce a cognitively inspired reinforcement learning method that can overcome this limitation by exploiting the hierarchical structure of human behavior. We leverage it to understand and improve human planning in large and complex sequential decision problems. Our method decomposes sequential decision problems into two sub-problems: setting a goal and planning how to achieve it. Our method can discover optimal human planning strategies for larger and more complex tasks than was previously possible. The discovered strategies achieve a better tradeoff between decision quality and computational cost than both human planning and existing planning algorithms. We demonstrate that teaching people to use those strategies significantly increases their level of resource-rationality in tasks that require planning up to eight steps ahead. By contrast, none of the previous approaches was able to improve human performance on these problems. These findings suggest that our cognitively informed approach makes it possible to leverage reinforcement learning to improve human decision-making in complex sequential decision problems. Future work can leverage our method to develop decision support systems that improve human decision-making in the real world.