Comparability of biological assessments derived from predictive models and multimetric indices of increasing geographic scope

The increasing demand for tools that can score biological condition from aquatic community data has spurred the creation of many predictive models (e.g., observed/expected [O/E] indices) and multimetric indices (MMIs). The geographic and environmental scopes of these indices vary widely, and coverages often overlap. If indices developed for large environmentally heterogeneous regions provide results equivalent to those developed for smaller regions, then regulatory entities could adopt indices developed for larger regions rather than fund the development of multiple indices within a region. We evaluated this potential by comparing the performance (precision, bias, responsiveness, and sensitivity) of benthic macroinvertebrate O/E indices and MMIs developed for California (CA) with that of indices developed for 2 large-scale condition assessments of US streams: the Environmental Monitoring and Assessment Program Western Pilot Study (EMAP-West) and the western portion of the Wadeable Streams Assessment (WSA-West). WSA-West and EMAP-West O/E scores were weakly correlated with CA O/E scores, had lower precision than CA O/E scores, were influenced by 2 related natural gradients (% slope and % fast-water habitat) that did not influence CA O/E scores, and disagreed with 21 to 22% of impairment decisions derived from the CA O/E index. The WSA-West O/E index produced many fewer impairment decisions than did the CA O/E index. WSA-West and EMAP-West MMI scores were strongly correlated with the CA MMI scores. However, the WSA-West and EMAP-West MMIs produced many fewer determinations of impairment than did the CA MMI. EMAP-West and WSA-West MMIs were biased and differed in responsiveness compared with CA MMI. Thus, they might produce estimates of regional condition different from those from indices calibrated to local conditions. The lower precision of the EMAP-West and WSA-West indices compromises their use in site-specific assessments where both precision and accuracy are important. However, the magnitude of differences in impairment decisions was sensitive to the thresholds used to define impaired conditions, so it might be possible to adjust some of the systematic differences among the models to make the large-scale models more suitable for local application. Future work should identify the geographic and environmental scales that optimize index performance, determine the factors that most strongly influence index performance, and identify ways to specify accurate reference condition from geographically extensive reference-site data sets