Testing long-term earthquake forecasts: likelihood methods and error diagrams

We propose a new method to test the performance of a spatial point process forecast based on a log-likelihood score for predicted point density and the information gain for events that actually occurred in the test period. The method largely avoids simulation use and allows us to calculate the information score for each event or set of events as well as the standard error of each forecast. As the number of predicted events increases, the score distribution approaches the Gaussian law. The degree of its similarity to the Gaussian distribution can be measured by the computed coefficients of skewness and kurtosis. To display the forecasted point density and the point events, we use an event concentration diagram or a variant of the Error Diagram (ED). We find forward relation between the error diagram curve and the information score as well as inverse relation for one simple model of point spatial fields. We again show that the error diagram is more informative than the likelihood ratio.We demonstrate the application of the method by using our long-term forecast of seismicity in two western Pacific regions. We compare the ED for these regions with simplified diagrams based on two-segment approximations. Since the earthquakes in these regions are concentrated in narrow subduction belts, using the forecast density as a template or baseline for the ED is a more convenient display technique. We also show, using simulated event occurrence, that some proposed criteria for measuring forecast effectiveness at EDs would be strongly biased for a small event number