Putting Small Area Estimation to the Test

This paper examines the performance of small area welfare estimation. The method combines census and survey data to produce spatially disaggregated poverty and inequality estimates. To test the method, predicted welfare indicators for a set of target populations are compared with their true values. The target populations are constructed using actual data from a census of households in a set of rural Mexican communities. Estimates are examined along three criteria: accuracy of confidence intervals, bias and correlation with true values. We find that while point estimates are very stable, the precision of the estimates varies with alternative simulation methods. While the original Elbers et al (2002, 2003) approach of numerical gradient estimation yields standard errors that seem appropriate, some computationally less-intensive simulation procedures yield confidence intervals that are slightly too narrow. Precision of estimates is shown to diminish markedly if unobserved location effects at the village level are not well captured in underlying consumption models. With well specified models there is only slight evidence of bias, but we show that bias increases if underlying models fail to capture latent location effects. Correlations between estimated and true welfare at the local level are highest for mean expenditure and poverty measures and lower for inequality measures