Autonomous Monitoring of a Diverse Ground Station Network

Planet Labs owns and operates the largest commercial earth-imagery CubeSat constellation. Planet’s ground station network is responsible for the earth-to-space communication link that gathers health and telemetry data, keeps the spacecraft schedule up-to-date, and downlinks the payload data from the spacecraft. The ground station network contains fifteen geographically diverse sites with a combination of leased and owned equipment from multiple vendors. Across those sites, the team monitors over 1600 services on nearly 500 devices. The scale of the network and diversity of equipment present challenges for operations and network health monitoring. Planet’s Ground Station Operations team monitors assets through a combination of active monitoring scripts on timers, event-based monitoring feedback, real-time metric analysis, and periodic automated long-term metric analysis. Active polling by monitoring scripts and real-time metric analysis catch configuration, software, and hardware issues as they arise independent of contacts with satellites and enable operators to quickly fix problems with little to no loss of satellite contact time. Meanwhile, event-based monitoring flags issues when outcomes differ from the expected results based on deterministic actions and uncover issues that are either transient or hidden from an active polling script. Last, long-term metric analysis gives insight into the slow degradation of system components and can be used to schedule targeted preemptive maintenance to efficiently maintain high operational uptime. With this combination of monitoring approaches and through using a wide array of tools that feed back into specific operator “dashboards” for a fast top-level view of issues, Planet’s Ground Station Operations team is able to maintain greater than 99% uptime and less than 90 minute incident response time without continuous 24-hour staffing. In total, the network takes over 2800 contacts per day with Planet’s Low Earth Orbit constellations. The Ground Station Operations team emphasizes automation, fail-over, and targeted redundancy to give on-call staff tools to rectify or triage issues quickly, efficiently, and at scale