Optimal Trade Execution Based on Deep Deterministic Policy Gradient

Abstract

In this paper, we address the Optimal Trade Execution (OTE) problem over the limit order book mechanism, which is about how best to trade a given block of shares at minimal cost or for maximal return. To this end, we propose a deep reinforcement learning based solution. Though reinforcement learning has been applied to the OTE problem, this paper is the first work that explores deep reinforcement learning and achieves state of the art performance. Concretely, we develop a deep deterministic policy gradient framework that can effectively exploit comprehensive features of multiple periods of the real and volatile market. Experiments on three real market datasets show that the proposed approach significantly outperforms the existing methods, including the Submit & Leave (SL) policy (as baseline), the Q-learning algorithm, and the latest hybrid method that combines the Almgren-Chriss model and reinforcement learning.