Ensembling forecast models with logistic regression

This repository is associated with the publication: Herrmann, M. and W. Marzocchi (2023). Maximizing the forecasting skill of an ensemble model. Geophysical Journal International. doi:10.1093/gji/ggad020 in which we ensemble short-term earthquake forecast models using multivariate logistic regression1: fitting the logistic model to various rate forecasts vs. the corresponding observations (target earthquakes). Besides creating a logistic model ensemble, we create a weighted-average ensemble based on mapping regression coefficients to weights. The Jupyter notebook demo-ensemble_OEF-Italy.ipynb (can also be viewed at nbviewer.jupyter.org) demonstrates our method for OEF-Italy2 by making use of the Python code in lr_ensemble.py (the class LREnsemble). While the logistic model ensemble performed poorly in this application3, the weighted-average ensemble is significantly more informative than the best candidate model (ETAS_LM)—even though the other candidates are overall less informative. In other words, our ensemble exploits their individual strengths. The approach exposes some level of flexibility to emphasize different aspects of seismicity and address different end users. Please find more information, applications, and findings in our paper. Footnote [1]: The logistic legression is implemented in scikit-learn (scikit-learn.org/.../sklearn.linear_model.LogisticRegression.html): Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Blondel M., Prettenhofer P., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. url: jmlr.csail.mit.edu/papers/v12/pedregosa11a.html Footnote [2]: OEF-Italy [Marzocchi et al. 2014] is an Operation Earthquake Forecasting system based on three forecasting models (ETAS_LM, ETES_FCM, STEP_LG). Marzocchi W., Lombardi A.M. & Casarotti E. (2014). The establishment of an operational earthquake forecasting system in Italy. Seismological Research Letters, 85(5), 961–969. doi: 10.1785/0220130219 Footnote [3]: Likely due to the combined effect of (i) short-term models changing their forecasts continuously (and occasionally significantly) and (ii) the sensitivity of the logistic regression intercept to very rare target events (∼0.006% in our case). ——————— Copyright © 2023 Marcus Herrmann · Warner Marzocchi — Università degli Studi di Napoli 'Federico II' Licensed under the European Union Public Licence (EUPL-1.2-or-later), the first European Free/Open Source Software (F/OSS) license. It is available in all official languages of the EU