An Investigation into the Construct Validity of an Academic Writing Test in English with Special Reference to the Academic Writing Module of the IELTS Test

The International English Language Testing System (IELTS) is the world’s leading high stakes test that assesses the English Language Proficiency of candidates who speak languages other than English and wish to gain entry into universities where English is the language of instruction. Recently, over 3000 institutions in the United States accepted the IELTS test to be an indicator of language proficiency (IELTS, 2012a). Because of this preference for the IELTS test, and its worldwide recognition, there has been an increase in the number of students who are taking the test every year. According to the IELTS website, more than 7000 institutions around the world trust the test results and, not surprisingly, more than 1.7 million candidates take the test every year in one of the 800 recognised test centres across 135 countries (IELTS, 2012a). These candidates include people who seek not only to obtain admission to universities, but also for immigration authorities, employers of certain companies and government agencies. Acknowledging this popularity and importance to learners of English as a Foreign Language (EFL), this qualitative study has investigated the construct validity of the academic writing module in the IELTS test from the perspectives of the stakeholders (i.e. candidates, lecturers and markers). The aim was to understand why some Saudi students fail to cope with demands of the university despite the fact that they have achieved the minimum requirements in IELTS. In this study, data was collected in two phases in two different settings through open-ended questionnaires, semi-structured observations and semi-structured interviews. Phase I was carried out in the Department of English Language (DEL) at King Faisal University in Saudi Arabia, while Phase II was conducted in one university in the UK. The sample of the study included: 8 students, 6 university lecturers and one marker. In this study, data were analysed and coded into themes by using NVivo 9. The results of this case study have shown that the stakeholders were doubtful about the issue of readiness of students, which is claimed by IELTS, and they wanted the test to be clearer about how the students were going to cope with university demands upon gaining entry. In addition, with respect to the content validity of the test, this study found that the tasks in the academic writing test to a large extent do not reflect the kind of tasks candidates are likely to encounter at university. Furthermore, this study pointed out that response validity, on the part of students who may not have understood the rubric of the tasks, is another important factor affecting the students’ performance. Also, the findings of this study suggested that scoring validity could have a significant effect on the students’ scores because of the inconsistency of markers during the scoring process as they may have sometimes failed to assign the students to their corresponding level of proficiency. Consequently, the study provided a set of implications as well as recommendations for future research.King Faisal University Ministry of Higher Education Kingdom of Saudi Arabi