Going Outside the Model: Robustness Analysis and Experimental Science

In 1966 the population biologist Richard Levins gave a forceful and in?uential defence of a method called “robustness analysis” (RA). RA is a way of assessing the result of a model by showing that different but related models give the same result. As Levins put it, “our truth is the intersection of independent lies” (1966, 423). Steven Orzack and Elliott Sober (1993) responded with an equally forceful critique of this method, concluding that the idea of robustness “lacks proper de?nition and that its bearing on the question of whether a proposition is true is highly problematic” (533). Replies to Orzack and Sober, from Levins (1993) and Weisberg (2006b), have rejected the idea that RA, on its own, can con?rm the results of models. I argue that these replies have not properly addressed Orzack and Sober’s real criticisms, which focus not on the role of empirical data in RA but on the problem of ensuring that the models used in RA are independent. By drawing on accounts of RA in experimental science, I argue that there is in fact a fallible but viable form of RA that can both con?rm the results of models and incorporate empirical data. However, for reasons other than those of Orzack and Sober, I conclude that RA may be of limited use in model-based science, as the example of ?shery biology can show