Planning with abstract Markov decision processes

40Citations
Citations of this article
86Readers
Mendeley users who have this article in their library.

Abstract

Robots acting in human-scale environments must plan under uncertainty in large state-action spaces and face constantly changing reward functions as requirements and goals change. Planning under uncertainty in large state-action spaces requires hierarchical abstraction for efficient computation. We introduce a new hierarchical planning framework called Abstract Markov Decision Processes (AMDPs) that can plan in a fraction of the time needed for complex decision making in ordinary MDPs. AMDPs provide abstract states, actions, and transition dynamics in multiple layers above a base-level "flat" MDP. AMDPs decompose problems into a series of subtasks with both local reward and local transition functions used to create policies for subtasks. The resulting hierarchical planning method is independently optimal at each level of abstraction, and is recursively optimal when the local reward and transition functions are correct. We present empirical results showing significantly improved planning speed, while maintaining solution quality, in the Taxi domain and in a mobile-manipulation robotics problem. Furthermore, our approach allows specification of a decision-making model for a mobile-manipulation problem on a Turtlebot, spanning from low-level control actions operating on continuous variables all the way up through high-level object manipulation tasks.

Cite

CITATION STYLE

APA

Gopalan, N., Desjardins, M., Littman, M. L., Macglashan, J., Squire, S., Tellex, S., … Wong, L. L. S. (2017). Planning with abstract Markov decision processes. In Proceedings International Conference on Automated Planning and Scheduling, ICAPS (pp. 480–488). AAAI press. https://doi.org/10.1609/icaps.v27i1.13867

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free