Assessing the Effect of Calibration on Nonresponse Bias in the 2006 ARMS Phase III Sample Using Census 2002 Data

The United States Department of Agriculture’s National Agricultural Statistics Service (NASS) conducts the annual Agricultural Resource Management Survey (ARMS) in three phases. The third phase of the ARMS collects detailed economic data which is highly sensitive. As a consequence, this phase suffers from relatively low response rates for a federal survey. According to the 2006 Office of Management and Budget Standards and Guidelines for Statistical Surveys, response rates lower than 80 percent may not only result in nonresponse bias, but they can jeopardize the future of surveys carried out by federal agencies. NASS has been operating under the assumption that the use of calibrated weights derived from appropriate targets addresses nonresponse bias in the 2006 Phase III ARMS. This assumption was tested using Census 2002 expenditure-sample data. The results showed that calibrated weights decreased bias levels so that they were no longer significantly different from zero at the 0.05 level for over 90 percent of the variables evaluated