SmartSat Constellation - A Deep Reinforcement Learning Approach for Decentralized Coordination

With rapid advancements in satellite technology, the amount of low earth orbit satellites has grown significantly which are primarily deployed for weather monitoring, earth observation or military purposes. Due to this reason, there has been an increased interest in enhancing the level of autonomy and cognition, onboard satellites to achieve optimal data collection. Optimal data is said to be collected when the satellites in a small sat constellation work together to collect information. This means that even if one of the satellites has missed out on some important information, the others can still collect them. A satellite constellation can be considered as a multi-agent reinforcement learning system. Having these agents coordinate with one another, can reduce the amount of time required to perform a task. The state-of-the-art satellite constellations follow a centralized coordination mechanism in which one primary satellite controls the rest of the satellites. This process is computationally more expensive and requires substantial communication between the satellites.It has a single point of failure and communication might be affected if the primary satellite fails. On the other hand, decentralized coordination allows agents to control their behavior themselves without the command of a supervised master. In this case, there is less inter-satellite communication which reduces the requirement for specialized onboard computational hardware. The proposal constitutes leveraging the Multi-Agent Deep Deterministic Policy Gradient [2] (MADDPG) algorithm to train the agents (satellites) to achieve optimal data collection. There are multiple use cases for the proposed solution such as illegal maritime activity tracking, natural disaster detection and assessing building damage after a natural disaster. The proposed solution focuses on tracking of ships in an extensively simulated environment for which a custom ship environment was created by leveraging OpenAI Gym [12]. By providing on-board autonomy, we aim to reduce frequent Earth Station (ES) communication significantly and enhance data collection capability