Measuring cognitive task demands using dual task methodology, subjective self-ratings, and expert judgments:a validation study

This study explored the usefulness of dual-task methodology, self-ratings, and expert judgements in assessing task-generated cognitive demands as a way to provide validity evidence for manipulations of task complexity. The participants were 96 students and 61 ESL teachers. The students, 48 English native speakers and 48 ESL speakers, carried out simple and complex versions of three oral tasks – a picture narrative, a map task, and a decision-making task. Half of the students completed the tasks under a dual task condition. The remaining half performed the tasks under a single task condition without a secondary task. Participants in the single condition were asked to rate their perceived mental effort and task difficulty. The ESL teachers provided expert judgments of anticipated mental effort and task difficulty along with explanations for their ratings via an online questionnaire. As predicted, the more complex task versions were found and judged to pose greater cognitive effort on most measures