Data Comparability in a Mixed Mode Telephone and Face to Face Survey of Persons with Disabilities

The purpose of this analysis is to compare data collected via telephone and face-to-face interviewing on a national survey of disability beneficiaries. Krosnick (2002) and others have suggested that telephone interviewing may pose more of a cognitive burden on respondents than face-to-face interviewing and that this effect may be more pronounced for individuals with reduced cognitive abilities. Since the sample for this survey is comprised entirely of individuals with physical and mental impairments, we were interested in whether survey mode would have an impact on data quality indictors such as acquiescence, social desirability, item non-response, and non-differentiation in response. To determine whether there were differences in data quality between telephone and face-to-face modes, chi-square analyses were performed to determine whether there was an association between mode and response for select items. Since participants were not randomly assigned to mode, we controlled for the non-experimental nature of the study by comparing subsets of respondents matched by demographic characteristics on several indicators of data quality. The results indicate that for the key items tested, there was more evidence of acquiescence, more social desirability in response, and more item non-response in CATI than CAPI. We conclude that mode of data collection may have effected data quality in this survey. However, we found that this effect was greatest for items that were vague or cognitively demanding. Factual questions about behavior appear to show the minimal, if any, mode effects