Performance-based measurement of national crisis management

Voyager was the biggest crisis management exercise ever held in The Netherlands. About 2000 professionals, ranging from fire-fighters to ministers and from local teams to the national crisis centre, participated in a complex multi-threat scenario. In order to learn from this exercise, an evaluation was carried out. The individual actors as such were not evaluated, but rather the overarching system of national crisis management. At the system level, three major crisis management functions were described: information exchange, decision-making, and communication with the media. Next, an evaluation format was developed. For each function, goals, criteria and performance indicators were identified. The performance indicators were specified in consultancy with representatives of all major actors during the Voyager exercise. This provided relevant information for the evaluators and made the results of the evaluation study more acceptable to the actors. Based on the evaluation format, observation schemes were designed to support the observers. Overall, 34 trained observers monitored the performance of 28 teams. The evaluation team combined all these observations and additional material into one coherent system evaluation. This paper describes the system approach to the evaluation, the evaluation format, (briefly) the main results of the Voyager exercise, and recommendations for further research