A standardized framework for evaluating the skill of regional climate downscaling techniques

Regional climate impact assessments require high-resolution projections to resolve local factors that modify the impact of global-scale forcing. To generate these projections, global climate model simulations are commonly downscaled using a variety of statistical and dynamical techniques. Despite the essential role of downscaling in regional assessments, there is no standard approach to evaluating various downscaling methods. Hence, impact communities often have little awareness of limitations and uncertainties associated with downscaled projections. To develop a standardized framework for evaluating and comparing downscaling approaches, I first identify three primary characteristics of a distribution directly relevant to impact analyses that can be used to evaluate a simulated variable such as temperature or precipitation at a given location: (1) annual, seasonal, and monthly mean values; (2) thresholds, extreme values, and accumulated quantities such as 24h precipitation or degree-days; and (3) persistence, reflecting multi-day events such as heat waves, cold spells, and wet periods. Based on a survey of the literature and solicitation of expert opinion, I select a set of ten statistical tests to evaluate these characteristics, including measures of error, skill, and correlation. I apply this framework to evaluate the skill of four downscaling methods, from a simple delta approach to a complex asynchronous quantile regression, in simulating daily temperature at twenty stations across North America. Identical global model fields force each downscaling method, and the historical observational record at each location is randomly divided by year into two equal parts, such that each statistical method is trained on one set of historical observations, and evaluated on an entirely independent set of observations. Biases relative to observations are calculated for the historical evaluation period, and differences between projections for the future. Application of the framework to this broad range of downscaling methods and locations is successful in that: (1) the downscaling method used is identified as a more important determinant of data quality than station location or GCM; and (2) key differences between downscaling methods are made apparent. For tests focusing on the general distribution of the variable, all methods except bias correction are relatively successful in simulating observed climate, suggesting that if an impact is most sensitive to changes in the mean, even a relatively simple downscaling approach such as ???delta??? will significantly improve simulation of local-scale climate. For tests that focus on the tails of the distribution, however, differences do arise between simple vs. quantile-based downscaling methods. Specifically, the latter appears less sensitive to location and more consistently able to reproduce observed climate. In terms of future projections, the most notable differences between downscaling methods becomes apparent at the right-hand tail of the distribution, where simple methods tend to simulate much greater increases (up to double the extreme heat days, for some locations) than more complex downscaling methods. I conclude by discussing how a standardized evaluation framework may advance our understanding of regional climate impact studies in understanding biases and limitations in results, as well as providing critical input into the selection of downscaling methods for future assessments. Given the potential exhibited by this initial test, I explore how this evaluation framework could be expanded in the future to make it even more useful: to the regional scale, for example, by including tests for spatial correlations and forcing relationships; or across variables, to capture interactions directly relevant to impact studies, such as heat waves (a function of temperature and humidity, affecting human health, energy demand, and agriculture) or snow amounts (a function of precipitation and temperature, affecting infrastructure and ecosystems); or to evaluate a broader selection of climate variables, downscaling methods, and predictor fields