Application of Deep Reinforcement Learning for Attitude Control of a Satellite in the Presence of Uncertainties

This paper investigates the performance of satellite attitude controllers based on deep reinforcement learning, trained in idealised simulation environments and deployed into uncertain and noisy simulation environments. Additionally, it is investigated whether training directly in the uncertain environment improves performance when deployed to that environment. Uncertainties considered are Gaussian white noise superimposed onto sensor measurements of the satellite angular velocity and attitude quaternion and uncertainty in the satellite inertia tensor. The platform selected is a 6U CubeSat. The results indicate that the deep-reinforcement-learning-based attitude controller is able to maintain pointing accuracy on satellites of different inertia tensor to the training satellite. The pointing accuracy decreases when the sensor measurements are subject to Gaussian white noise. The results also suggest that training directly in the environment subject to these uncertainties does not improve pointing accuracy relative to the controller trained in the ideal environment if these uncertainties cause the state vector to break the Markov property. Furthermore, training in the uncertain environment can add instability to the learning process and prevent the controller from converging to a high performance behavioural policy