How to evaluate an early warning system: Toward a unified statistical framework for assessing financial crises forecasting methods

This paper proposes an original and unified toolbox to evaluate financial crisis early-warning systems (EWS). It presents four main advantages. First, it is a model free method which can be used to assess the forecasts issued from different EWS (probit, logit, Markov switching models, or combinations of models). Second, this toolbox can be applied to any type of crisis EWS (currency, banking, sovereign debt, and so on). Third, it does not only provide various criteria to evaluate the (absolute) validity of EWS forecasts but also proposes some tests to compare the relative performance of alternative EWS. Fourth, this toolbox can be used to evaluate both in-sample and out-of-sample forecasts. Applied to a logit model for 12 emerging countries we show that the yield spread is a key variable for predicting currency crises exclusively for South-Asian countries. Besides, the optimal cut-off correctly allows us to identify now on average more than 2/3 of the crisis and calm periods. [JEL C33, F37] IMF Economic Review (2012) 60, 75-113. doi: 10.1057/imfer.2012.4