Language-based General Action Template for Reinforcement Learning Agents

Abstract

Prior knowledge plays a critical role in decision-making, and humans preserve such knowledge in the form of natural language (NL). To emulate real-world decision-making, artificial agents should incorporate such generic knowledge into their decision-making framework through NL. However, since policy learning with NL-based action representation is intractable due to NL's combinatorial complexity, previous studies have limited agents' expressive power to only a specific environment, which sacrificed the generalization ability to other environments. This paper proposes a new environment-agnostic action framework, the language-based general action template (L-GAT). We design action templates on the basis of general semantic schemes (FrameNet, VerbNet, and WordNet), facilitating the agent in finding a plausible action in a given state by using prior knowledge while covering broader types of actions in a general manner. Our experiment using 18 text-based games showed that our proposed L-GAT agent which uses the same actions across games, achieved a performance competitive with agents that rely on game-specific actions. We have published the code at https://github.com/kohilin/lgat.