Bootstrap-based criteria for choosing the number of instruments

Instrumental variables estimation is a widely used technique in many fields in the social sciences. An important problem in applying the technique is the choice of instrumental variables, since empirical researchers often have many (valid) instrumental variables from which to choose. According to conventional asymptotic theory, the asymptotic variance of an estimator decreases with the number of instruments used for estimation. However, instrumental variables estimators behave poorly when there are many moment conditions. In particular, the two-stage least squares (2SLS) estimator, which is the most widely used instrumental variables estimator, has a bias that is proportional to the number of instruments. Appropriate selection of the number of instruments is therefore essential for obtaining good estimates. The main purpose of this paper is to report the results of Monte Carlo simulations conducted to investigate the ability of bootstrap based criteria for choosing a number of instrumental variables. We consider the 2SLS estimation of a linear simultaneous equations model. The bootstrap is used to estimate the mean square error (MSE) of the estimator, and then the number of instruments is chosen by minimizing the bootstrapped MSE. The bootstrap is a method for approximating the distribution of a statistic by computing the distribution of that statistic under an estimated distribution of the data (which is often the empirical distribution of the data). In practice, we repeatedly draw samples from an estimated distribution of the data and compute the statistics of interest for each sample. The empirical distributions of the statistics computed in this way are used as estimates of the distributions of the statistics. This paper considers three different bootstrap procedures. The first is the "naive" bootstrap, in which each bootstrap sample is drawn from the empirical distribution of the data. In the second bootstrap procedure, bootstrap samples are drawn from the empirical distribution, which is modified so that the instruments are orthogonal to the residuals of the equation. The third bootstrap procedure involves an analytical bias correction. When we compute the bootstrapped MSE, we use an analytic approximation of the bias rather than the bootstrap bias estimate. The results of our experiments show that the "naive" bootstrap does not work for the purpose of choosing instruments. A large number of instruments tend to be chosen by the "naive" bootstrap method. When the sample size is small, the estimator based on the choice of instruments implied by the "naive" bootstrap performs similarly to the estimator based on the use of all available instruments. When the bootstrap sample is drawn so that the instruments are orthogonal to the residuals, the estimator is more precise, but it does not perform well in small samples. The problem is that the bootstrap estimate of the bias is inaccurate. By correcting the bias by using an analytic approximation of the bias rather than the bootstrap bias estimate, the bootstrap-based procedure improves the performance of the estimator. With these modifications, the bootstrap procedure performs equally as well, and often better, than the selection method of Donald and Newey (2001)