Critical appraisal in environmental evidence synthesis: challenges and options

Critical appraisal is a crucial step in systematic reviews that, when conducted appropriately, enables threats to the validity of evidence to be identified and accounted for in the data synthesis. However, critical appraisal in published environmental management systematic reviews, and those submitted to the Collaboration for Environmental Evidence, is often conducted inconsistently, does not always focus on threats to validity, and does not always inform the data synthesis. The validity of conclusions from many environmental systematic reviews may therefore be questionable, but there is a lack of reliable tools to guide consistent critical appraisal of environmental research studies. This presentation will discuss a conceptual model-based framework that could improve the consistency and transparency of critical appraisal. It aims to guide review teams on how to identify different threats to validity (e.g. risks of bias) in primary research studies, and has flexibility for application alongside existing critical appraisal tools, where available