A Deep Q-Learning Direct Torque Controller for Permanent Magnet Synchronous Motors

Abstract

Torque control of electric drives is a challenging task, as high dynamics need to be achieved despite different input and state constraints while also pursuing secondary objectives, e.g., maximizing power efficiency. Whereas most state-of-the-art methods generally necessitate thorough knowledge about the system model, a model-free deep reinforcement learning torque controller is proposed. In particular, the deep Q-learning algorithm is utilized which has been successfully used in different application scenarios with a finite action set in the recent past. This nicely fits the considered system, a permanent magnet synchronous motor supplied by a two-level voltage source inverter, since the latter is a power supply unit with a limited amount of distinct switching states. This contribution investigates the deep Q-learning finite control set framework and its design, including the conception of a reward function that incorporates the demands concerning torque tracking, efficiency maximization and compliance with operation limits. In addition, a comprehensive hyperparameter optimization is presented, which addresses the many degrees of freedom of the deep Q-learning algorithm striving for an optimal controller configuration. Advantages and remaining challenges of the proposed algorithm are disclosed through an extensive validation, which includes a direct comparison with a state-of-the-art model predictive direct torque controller.