Model selection and sloppiness in ecology

In ecology, model selection is important for making sure that models used for conservation and management decisions are not overly impaired by model structure uncertainty. However, the most widely used model selection metrics in ecological modelling, such as the Akaike information criterion (AIC) or the Bayesian information criterion (BIC), utilise simple measures for penalising complexity that heavily rely on asymptotic results for large sample sizes (Gelman et al., 2014). While these standard complexity measures are straightforward to implement, they do not account for the complex dependencies between model parameters that are particularly common in ecological models. Analyses of both model sloppiness and parameter identifiability can provide insights into a key challenge for practical model selection, that is, the quantification of multi-parameter model complexity (Browning et al., 2020; Transtrum et al., 2015). Both analyses, based on spectral properties of the Fisher Information Matrix, seek to characterise the sensitivity of the model outputs to changes in parameter values and thus can reveal critical combinations of parameters, weighted by their influence on model outputs. These weighted combinations in parameter space provide scope for an intrinsic measure for multi-parameter model complexity, which can be exploited for model selection, model reduction, and even future data collection efforts. In this work, we demonstrate the connection between the principles of model sloppiness and the Laplace approximation of the Bayesian posterior model probability that reduces to the BIC in the large sample limit (Neath and Cavaneugh, 2012). This link has not been previously identified; however, this non-asymptotic form has potential benefit in the all-too-common case where very few noisy observations are available to calibrate complex models possessing many parameters requiring estimation. To this end, we explore the utility of model sloppiness analysis in combination with information criteria for robust model selection when models have realistic sample sizes for ecological time-varying datasets. Through simulations, we investigate the performance of these methods for ecological models based on ordinary differential equations, highly relevant in ecological conservation to guide ecosystem management and risk assessment. We highlight several important scenarios where accounting for model sloppiness detects the true model more frequently for small numbers of observations. As a result, this work provides practical guidelines for using model selection metrics in the landscape of ecological models and demonstrates that accounting for model sloppiness and parameter identifiability is essential for realistic applications in ecology