Designing Sun–Earth L2 Halo Orbit Stationkeeping Maneuvers via Reinforcement Learning

Abstract

Reinforcement learning (RL) is used to design impulsive stationkeeping maneuvers for a spacecraft operating near an L2 quasi-halo trajectory in a Sun–Earth–Moon point mass ephemeris model with solar radiation pressure. This scenario is translated into an RL problem that reflects the desired stationkeeping goals, variables, and dynamical model. An algorithm from proximal policy optimization is used to train a policy that generates stationkeeping maneuvers while transfer learning is used to reduce the computational time required for training. The trained policy successfully generates stationkeeping maneuvers that result in boundedness to the vicinity of the selected reference trajectory with low total maneuver requirements, producing comparable results to a traditionally formulated constrained optimization scheme.