Assessing annual global M6+ seismicity forecasts

We consider a seismicity forecast experiment conducted during the last 4 yr. At the beginningof each year, three models make a 1-yr forecast of the distribution of large earthquakeseverywhere on the Earth. The forecasts are generated and the observations are collected in theCollaboratory for the Study of Earthquake Predictability (CSEP). We apply CSEP likelihoodmeasures of consistency and comparison to see how well the forecasts match the observations,and we compare results from some intuitive reference models. These results illustrate someundesirable properties of the consistency tests: the tests can be extremely sensitive to only afew earthquakes, and yet insensitive to seemingly obvious flaws-a näive hypothesis that largeearthquakes are equally likely everywhere is not always rejected. The results also suggest thatone should check the assumptions of the so-called T and W comparison tests, and we illustratesome methods to do so. As an extension ofmodel assessment, we explore strategies to combineforecasts, and we discuss the implications for operational earthquake forecasting. Finally, wemake suggestions for the next generation of global seismicity forecast experiments.©The Authors 2013