Randomness and Evaluation in Research Funding

Which role do randomness and evaluation play in research funding. Can peers give reliable estimation of the future scientific success of grant applications? Which functions can lotteries fulfill in the future? In their pivotal study of 1981, Cole et al. found that 50% of funding decisions were due to the (random) selection of reviewers. This raises fundamental questions on efficacy and legitimacy of peer review decisions in science funding. Empirical studies trying to discover the association between scientific performance and evaluation from peer review found conflicting evidence (Berg, 2011a, 2011b; Danthi Narasimhan et al., 2014; Lauer Michael S. et al., 2015; Li & Agha, 2015). At times the debate on the introduction of lotteries centers around a cost-benefit logic, even though potential gains in efficiency through reduction of workload for grant applicants and reviewers still remain unproven (Liu et al. 2020) and potential losses on the benefit side have to be considered, too (Reinhart & Schendzielorz, 2020)