Reproducible science case studies in ecology: Lessons learned

Some say science is in a reproducibility crisis, meaning that it is almost impossible to reproduce published research results. Complete reproducibility is a lofty goal but a continuum of practices provides achievable steps towards that goal. Clearly, most ecological field sampling may not be reproduced due to variability among sites and changing environmental conditions over time. Hence, steps towards reproducibility generally start with published data, which may be improved with documentation and published data processing and analytical code. This culminates in a complete workflow (raw data to published results) being publicly available for each scientific publication. We present three pioneering case studies of ecological research and introduce tools to achieve documentation of a complete data analysis workflow. The projects are (1) a small group project using publicly available raw data, (2) a larger project which set out with no plans for reproducibility, and (3) a multi-investigator Macrosystems project which developed effective open collaboration policies at the outset and tested tools and approaches to documenting all data manipulations.Funding was provided by NSF ABI 126245