Generalizability of artificial neural network models in ecological applications: Predicting nest occurrence and breeding success of the red-winged blackbird Agelaius phoeniceus

Separate artificial neural network (ANN) models were developed from data in two geographical regions and years apart for a marsh-nesting bird, the red-winged blackbird Agelaius phoeniceus. Each model was independently tested on the spatially and temporally distinct data from the other region to determine how generalizable it was. The first model was developed to predict occurrence of nests in two wetlands on Lake Erie, Ohio in 1995 and 1996. The second model was developed to predict breeding success in two marshes in Connecticut, USA in 1969 and 1970. Independent variables were vegetation durability stem density, stem/nest height, distance to open water, distance to edge, and water depth. The nest occurrence model performance on the training data were at an average cross entropy, or concordance index (c-index), of 0.75. Within geographical region testing in two different wetlands resulted in c-indices of 0.66 and 0.53. The breeding success model performance was at ac-index of 0.75 on the training data and at c-indices of 0.47 and 0.53 for within region testing. When we tested the nest occurrence model on fledged nestling data we obtained c-indices of 0.69 and 0.47 in Clarkes Pond in 1969 and 1970, respectively, and 0.43 and 0.52 in All Saints Marsh in 1969 and 1970, respectively When we tested the fledged nestling model on the nest occurrence data, we obtained c-indices of 0.70 and 0.41 in Stubble Patch in 1995 and 1996, respectively, and 0.54 and 0.55 for Darr in 1995 and 1996, respectively With input variable relevances, sensitivity analyses and neural interpretation diagrams we were able to understand how the different models predicted nest occurrence and breeding success and compare their differences and similarities. important variables for predicting nest site selection/breeding success in both models were vegetation durability and distance to open water. Both models also predicted increasing nest occurrence/breeding success with increasing water depth under the nest and increasing distance to edge. However, relationships for prediction differed in the models. Generalizability of the models was poor except when the marshes had similar values of important variables in the model, for example water depth. ANN models performed better than generalized linear models (GLM) on marshes with similar structures. Generalizability of the models did not differ in nest occurrence and breeding success data. Extensive testing also showed that the GLMs were not necessarily more generalizable than ANNs. The results from this study suggest that ANN models make good definitions of a study system but are too specific to generalize well to other ecologically complex systems unless input variable distributions are very similar. (c) 2005 Published by Elsevier B.V