Machine learning for scientific data mining and solar eruption prediction

This dissertation explores new machine learning techniques and adapts them to mine scientific data, specifically data from solar physics and space weather studies. The dissertation tackles three important problems in heliophysics: solar flare prediction, coronal mass ejection (CME) prediction and Stokes inversion. First, the dissertation presents a long short-term memory (LSTM) network for predicting whether an active region (AR) would produce a certain class of solar flare within the next 24 hours. The essence of this approach is to model data samples in an AR as time series and use LSTMs to capture temporal information of the data samples. The LSTM network consists of an LSTM layer, an attention layer, two fully connected layers and an output layer. The attention layer is designed to allow the LSTM network to automatically search for parts of the data samples that are related to the prediction of solar flares. Second, the dissertation presents two recurrent neural networks (RNNs), one based on gated recurrent units and the other based on LSTM, for predicting whether an AR that produces a significant flare will also initiate a CME. Again, data samples in an AR are modeled as time series and the RNNs are used to capture temporal dependencies in the time series. A feature selection technique is employed to enhance prediction accuracy. Third, the dissertation approaches the Stokes inversion problem using a novel convolutional neural network (CNN). This CNN method is faster, and produces cleaner magnetic maps, than a widely used physics-based tool. Furthermore, the CNN method outperforms other machine learning algorithms such as multiple support vector regression and multilayer perceptrons. Findings reported here have been validated by substantial experiments based on different datasets. The dissertation concludes with a fully operational database system containing real-time flare forecasting results produced by the proposed LSTM method. This is the first cyberinfrastructure capable of continuous learning and forecasting of solar flares based on deep learning