Countering Method Bias in Questionnaire-Based User Studies

Purpose This paper discusses reliability problems associated with questionnaires, commonly employed in library and information science. It focuses on the effects of “common method variance ” (CMV), which is a form of bias, and ways of countering these effects. Approach We critically review the use of existing tools for demonstrating reliability in questionnaire-based studies. In particular we focus on Cronbach’s alpha, “Harman’s single factor test ” and Lindell and Whitney’s “marker variable ” approach. We introduce an illustrative case study based on our own work on metacognition and web searching. We go on to make recommendations for researchers considering using a questionnaire-based approach. Findings CMV is a problem affecting questionnaire-based studies in different disciplines across social and information science. Where questionnaire items are more abstract, CMV ha