You are here

Reproducibility in Ecological Research

Tuesday, January 16, 2018
Printer-friendly version
Figure from Nature Ecology and Evolution

How can we improve the reproducibility of ecological experiments? In a paper published this January in Nature Ecology and Evolution, an international team led by Alex Milcu that included Harvard Forest Senior Ecologist Aaron Ellison shows that a deliberate introduction of controlled systematic variability (CSV) in experimental designs may increase the reproducibility of experiments conducted at multiple sites or in multiple laboratories.

One would expect that if one group tried to reproduce the experimental results of another group, then reproducing each methodological step would be critical - using, for example, identical seed sources, soils, and reagents. In contrast, Milcu and his team tested the controversial hypothesis that such stringent levels of environmental and biotic standardization actually reduce reproducibility by amplifying the impacts of laboratory-specific environmental factors not accounted for in study designs. This hypothesis suggests that deliberately introducing “controlled systematic variability” (CSV) into an experiment should increase the likelihood that the experiment can be reproduced.

To test this hypothesis, 14 European laboratories ran a simple microcosm experiment using grass (Brachypodium distachyon) monocultures and grass + legume (Medicago truncatula) mixtures. Each laboratory introduced environmental, genotypic, or both environmental and genotypic CSV in either growth chambers or glasshouses.

Introducing genotypic CSV increased reproducibility in growth chambers, which have stringent environmental controls, but not in glasshouses, which do not. In contrast, introducing environmental CSV had little effect on reproducibility in either growth chambers or glasshouses. Researchers and others have suggested multiple causes for the ‘reproducibility crisis’ in science, and ecologists have suggested that reproducibility in ecological studies may be particularly difficult. Milcu's team's results show that the deliberate introduction of known, quantified genetic variability may be a simple way to increase the reproducibility of ecological studies done in stringently controlled environmental conditions.

Content Tags: