Do we see through their eyes? Testing a bilingual questionnaire in education research using cognitive interviews

The paper aims to demonstrate the value of cognitive interviewing (CI) as a survey pretesting method in comparative education research. Although rarely used by education researchers, CI has been successfully applied in different disciplines to evaluate and improve question performance. The method assumes that observing people’s thought processes when they answer survey questions can detect response problems and point to possible solutions. To illustrate the merits of CI, we present the findings from eight cognitive interviews, which informed the development of a bilingual English/Georgian online questionnaire. The main objectives of our CI study were to a) examine cognitive validity of survey questions, b) determine semantic equivalence of the source (English) and translated (Georgian) versions of the questionnaire, and c) establish conceptual equivalence of survey measures across two cultures. We conducted two rounds of cognitive interviews, one in each language, using a combination of think-aloud and verbal probing techniques. Our analysis suggests that CI can help to identify causes of response difficulties and develop more accurate and comparable survey measures for cross-cultural education research