Monitoring of overshooting tops over East Asia using Himawari-8 satellite data and machine learning approaches

Overshooting tops (OTs) are extreme convective cloud tops with strong updraft penetrating into the lower stratosphere. They transport surface thermal energy to the higher troposphere as well as lower stratosphere through updraft air mass. OTs are typically related to surrounding atmospheric conditions, often resulting in extreme weather events such as lightnings, hails, and heavy rainfall. Water vapors penetrating into the lower stratosphere generate air turbulence which could be a serious threat to flight operation. Consequently, some OT detecting algorithms such as brightness temperature differences (BTDs) based detection (Schmetz et. al., 1997; Setvak et.al., 2007), infrared window(IRW)-texture based algorithm (Bedka et.al., 2010) and infrared (IR)/ visible (VIS) rating based probabilistic classification method (Bedka and Khlopenkov, 2016) have been proposed for detecting OTs. However, the existing algorithms are generally threshold-based algorithms, which often result in high false alarms. Bedka et al. (2016) tried to reduce false alarms by adopting a visible rating approach, but the detection rate a bit decreased as well. In this study, we used two machine learning approaches for OT detection to evaluate their performance to see if they result in high detection rates with a significant reduction in false alarms. The proposed OT detection methods consist two parts: 1) two machine learning approaches—aggregated decision trees(ADT) and logistic regression (LR)— based automatic OT classification, and 2) post-processing to reduce false alarms. Himawari-8 data (i.e. 16 channel multispectral images every 10 minutes with a resolution of 2 km to 500 m) over East Asia are used in this study. OT reference data were collected using visible channel (0.64 µm) and infrared channel (11.2 µm) images along with MODIS 250 m images through visual interpretation based on the characteristics of OT shape (i.e. dome-like protrusion and anvil cloud with ring-like waveform). A total of 17 variables (i.e. brightness temperature (BT) at 11.2 µm, BT based standard deviations and a difference between the center and boundary pixels in a series of windows (3x3, 5x5, 7x7, 9x9, 11x11 pixels), and split windows) were extracted based on OT and nonOT references. These 17 variables are related to physical or spatial characteristics of OTs. While the LR approach develops a nonlinear regression model for classification, the ADT approach is a probabilistic determination model producing dozens of rule-based trees. The hindcast validation results over East Asia showed that ADT performed better than LR, resulting in an average probability of detection (POD) of 70.2 % and an average false alarm rate (FAR) of 49.4 %, while LR results in POD of 50.8 % and FAR of 80.7 %. Through the second post-processing step, which consists of a series of thresholds related to the spatial shape of OTs, FAR was reduced 49.4 % to 24.5 % with a slight reduction of POD from 70.2% to 65.8 %. Considering the performance of existing approaches, the proposed models are very promising in monitoring OTs for an operational purpose