Implications of the Regional Earthquake Likelihood Models test of earthquake forecasts in California

The Regional Earthquake Likelihood Models (RELM) test was the first competitive comparison of prospective earthquake forecasts. The test was carried out over 5 years from 1 January 2006 to 31 December 2010 over a region that included all of California. The test area was divided into 7682 0.1°x0.1° spatial cells. Each submitted forecast gave the predicted numbers of earthquakes <em>N_emi</em> larger than <em>M</em>=4.95 in 0.1 magnitude bins for each cell. In this paper we present a method that separates the forecast of the number of test earthquakes from the forecast of their locations. We first obtain the number <em>N_em</em> of forecast earthquakes in magnitude bin <em>m</em>. We then determine the conditional probability <em>λ_emi</em>=<em>N_emi/</em><em>N_em</em> that an earthquake in magnitude bin <em>m</em> will occur in cell <em>i</em>. The summation of <em>λ_emi</em> over all 7682 cells is unity. A random (no skill) forecast gives equal values of <em>λ_emi</em> for all spatial cells and magnitude bins. The <em>skill</em> of a forecast, in terms of the location of the earthquakes, is measured by the success in assigning large values of <em>λ_emi</em> to the cells in which earthquakes occur and low values of <em>λ_emi</em> to the cells where earthquakes do not occur. Thirty-one test earthquakes occurred in 27 different combinations of spatial cells <em>i</em> and magnitude bins <em>m</em>, we had the highest value of <em>λ_emi</em> for that <em>mi</em> cell. We evaluate the performance of eleven submitted forecasts in two ways. First, we determine the number of <em>mi</em> cells for which the forecast <em>λ_emi</em> was the largest, the best forecast is the one with the highest number. Second, we determine the mean value of <em>λ_emi</em> for the 27 <em>mi</em> cells for each forecast. The best forecast has the highest mean value of <em>λ_emi</em>. The success of a forecast during the test period is dependent on the allocation of the probabilities λemi between the mi cells, since the sum over the mi cells is unity. We illustrate the forecast distributions of <em>λ_emi</em> and discuss their differences. We conclude that the RELM test was successful in illustrating the choices required when a forecast of the location of a future earthquake is made