What were They Thinking? Using Cognitive Interviewing to Examine the Validity of Self-Reported Epistemic Beliefs

We employed cognitive interviewing with a sample of secondary, college, undergraduate and graduate students to examine the cognitive validity of a popular epistemic beliefs self-report measure, the Discipline-Focused Epistemological Beliefs Questionnaire [DFEBQ] (Hofer, 2000). In addition, we examined cognitive validity across two domains. Analyses of interviews revealed that cognitive validity was good, wherein students ’ responses were typically within an expected range of interpretations. However, students ’ interpretations of items were not always consistent with researchers ’ intended meanings, interpretations sometimes differed across domains, and that the response option “3 ” as a neutral response was not always used as intended. To improve validity of self-report measures of epistemic beliefs more generally, we recommend that explicit anchors are used, such as “mathematician ” instead of “expert, ” and that definitions of the dimensions are presented to respondents to ensure interpretations align with researchers ’ intended meanings. We end with broader methodological implications