How to Best Apply Neural Networks in Geosciences: Towards Optimal Averaging in Dropout Training

The main objectives of geosciences is to find the current state of the Earth -- i.e., solve the corresponding inverse problems -- and to use this knowledge for predicting the future events, such as earthquakes and volcanic eruptions. In both inverse and prediction problems, often, machine learning techniques are very efficient, and at present, the most efficient machine learning technique is deep neural training. To speed up this training, the current learning algorithms use dropout techniques: they train several sub-networks on different portions of data, and then average the results. A natural idea is to use arithmetic mean for this averaging , but empirically, geometric mean works much better. In this paper, we provide a theoretical explanation for the empirical efficiency of selecting geometric mean as the averaging in dropout training