Testing the Untestable or Defending the Indefensible: An investigation into EAP assessment usefulness

With the Home Office permitting licenced Higher Education Institutions (HEIs) to decide how students’ English language ability is assessed, many institutions use in-house English for Academic Purposes (EAP) assessments developed to reflect the particular context and focus of the HEI in which they are used. These are often created administered by and further developed by EAP practitioners. For this development to take place, it is necessary to establish the usefulness of such assessments to decide what development is required. In an attempt to address gaps in the literature relating to research on in-house developed assessments, with a focus on how to establish whether these are useful to assess language proficiency, as well as providing students with the opportunity to develop wider academic skills, the research question for this study is: how can EAP practitioners establish the usefulness of in-use in-house EAP assessments? Bachman and Palmer’s (1996) test usefulness framework is used as a basis for developing a practical framework for evaluating in-use assessments. The literature review considers EAP, language assessment and academic acculturation, ending with an evaluative framework for establishing authenticity of tasks, their predictive validity, and reliability of grading criteria and marking schemes. The focus group case-study approach demonstrates how the framework can be given an institute-specific focus, considering opinions of three key stakeholder groups: EAP lecturers, subject-specialist lecturers and former pre-sessional students. Findings suggest academic language is highly contextual, and this context is institute- and, perhaps, department-specific. The findings suggest that for EAP assessments to be useful they need to support students with academic acculturation, providing valid and reliable results, supporting wider academic skills development. Using the evaluative framework and establishing the institutional context in which students find themselves, this case-study of the EAP pre-sessional assessments used between 2010 and 2014 culminates in a transferable approach which can be used by practitioners before considering further assessment development