Action-based representation discovery in markov decision processes

Abstract

Approaches for representation discovery in (S)MDPs using basis functions, function approximation techniques. Shows that these perform better than traditional hand-crafted approaches. Examined automatic building basis function representations by explicitly incorporating actions and doing this for hierarchical RL as well. State-action representations were shown to outperform state representations as it can simultaneously generalize over states and actions. Method proceeds by building graphs of transitions in the domain, and performing spectral analysis on these graphs. This captures the underlying structure of the domain. Also uses pre-defined task hierarchies.