Approximation for Bayesian ability estimation

An approximation is proposed for the posterior mean and standard devia-tion of the ability parameter in an item response model. The procedure assumes that approximations to the posterior mean and covariance matrix of item parameters are available. It is based on the posterior mean of a Taylor series approximation to the posterior mean conditional on the item parameters. The method is illustrated for the two-parameter logistic model using data from an ACT math test with 39 items. A numerical comparison with the empirical Bayes method using n = 400 examinees shows that the point estimates are very similar but the standard deviations under empirical Bayes are about 2 % smaller than those under Bayes. Moreover, when the sample size is decreased to n = 100, the standard deviation under Bayes is shown to increase by 14 % in some cases. The standard procedure for measuring ability in item response theory is to first estimate the item parameters for a set of test items and then estimate ability treating the item parameters estimates as known true values. Such practice ignores a source of uncertainty and leads to errors. The point of this paper is to present a Bayesian method of dealing with this uncertainty and to suggest something about the nature and magnitude of the inferential errors. We consider estimating the ability of an individual based on the person's dichotomous responses to a set of test items whose characteristics are partially known through responses to the same set from other individuals belonging to the same population. Each item is characterized by an item response function which defines the probability of a correct response to th