Understanding the performance of flood forecasting models for investment and incident management: Interim Report - SC130006/R1.

Understanding the performance of flood forecasting models - operated in real-time by the Environment Agency, Natural Resources Wales and the Flood Forecasting Centre - is essential to their informed use for flood guidance across England & Wales. It is also essential to guide future investment in flood incident management. This Interim Report presents progress on a project which aims to design a methodology for assessing flood forecasting models. The methodology is to be implemented to assess local forecast models operated as regional systems alongside the G2G distributed national forecast model configured to have coverage over England & Wales. The scope of work is outlined and project activities reviewed. Activities include an Inception Planning Workshop and engagement with supplier consultants of local forecast model reports on delivery of the underpinning datasets for use in the project. Review of a draft Framework and Proforma for assessment, proposed by the EA, provided useful guidance on requirements but identified the need for a fresh approach to implementation. This would focus on the collation of datasets of observed and forecast flow/level time-series, and not the performance measures themselves. The resulting database could then be subject to a consistent performance analysis producing agreed measures and outputs on performance, and open to change and addition to meet future requirements for assessment. Consideration of the forms of performance assessment to apply are discussed and reported on as an ongoing activity. This work aims to design and trial the overall performance assessment procedure, to agree the approach with the Project Board and then apply it to all datasets once received. The statistical methodology for performance assessment to be used is outlined, initially focussing on the calculation of categorical skill scores using an “all (available) forecast” approach rather than one that is solely “event-based”. A prototype form of Performance Summary applicable to a given model and site - including site/model information, performance metric and event-hydrograph displays, and performance grading – is developed and trialled as a proposed way forward. Some remaining issues are raised for discussion. The Interim Report ends with a summary of achievements to date and an outline of Next Steps