Evaluating and Using Statistical Methods in the Social Sciences

Introduction The "Bayesian information criterion" (BIC) can be a helpful statistical tool in sociology and elsewhere (see Raftery, 1995, and Kass and Raftery, 1995, for discussion and examples). However, Weakliem (1998) presents several powerful criticisms, both theoretical and applied, of BIC, which are similar to critical issues discussed in Gelman and Rubin (1995). Weakliem's paper makes three main points. First, the "Bayesian information criterion" (BIC) corresponds to a specific probability model that, in many examples, does not make good scientific sense. Second, the mathematical form of BIC can lead to pathologies with respect to sample size, data grouping, and model formulation, as illustrated with several real examples in Weakliem's paper. Third, in the case of the social mobility data used as an example by Raftery (1986, 1995), using BIC to select a model leads to scientifically misleading conclusions. This third point is in fact a problem with any autom