Test impact as dynamic process: individual experiences of the English test requirements for permanent skilled migration in Australia

© 2017 Dr. Kellie FrostThe use of language tests in immigration policy contexts raises critical questions concerning the adequacy of current conceptualisations of test purpose, impact, and validity in the field of language testing. Within existing theoretical frameworks, these notions rest on the premise that intended test purpose and intended score meanings exist as coherent, fixed, and stable entities, as defined by test developers, and these intentions must be specified as a precursor to evaluations of consequences and validity. Moreover, within existing frameworks, test takers exist primarily as theoretical abstractions rather than as real persons, deconstructed into the components of knowledge and skills that constitute test constructs. Such underlying assumptions, it is argued in this thesis, have limited our capacity in the field of language testing to understand and explain the role of language tests in immigration and other policy domains, where score meanings, test purposes and test consequences are shaped by multiple and often conflicting interpretations and values, as stakeholders, including test takers, struggle to realise their own intentions. This thesis explores the dynamics of test impact in the context of Australia’s skilled migration policy by examining how four individuals responded to the English test score requirements that exist within the policy as they sought to transition from temporary to permanent resident status. Temporary visa holders in Australia are able to become permanent residents via the skilled migration program if they possess certain specified skills and attributes, including English language proficiency as demonstrated on an accredited language test, such as IELTS. Since 2011, test scores of IELTS 7 and 8 have been heavily weighted in the overall selection process for skilled migrants, and thus represent a vital transition mechanism in the trajectory of those seeking to move from the status of temporary to permanent resident in Australia. A series of in-depth, open-ended interviews were conducted over an 18-month period with each of the four individuals during their migration trajectory from temporary to permanent residency, a transition that for each of them involved repeated language test attempts. A grounded theory approach guided analysis of interview data, which focused on identifying how each person thought, felt and acted in response to the test requirements, how their perceptions and feelings shifted over time as they interacted with the test, and how their test experiences shaped their lives and influenced their actions and decisions as they sought to realise their migration intentions in this highly constrained policy space. Findings show that individual subjectivities and agency intersected with the constraints of the test and score requirements in the context of Australia’s skilled migration policy to generate diverse and multidirectional consequences in each of the four migrant cases, highlighting the indeterminable nature of test purpose and test impact in this policy domain, and supporting the argument made throughout the thesis that the perceptions and experiences of those subjected to testing regimes, rather than the intentions of test developers, must constitute the focal point around which to evaluate the appropriateness of testing practices in policy domains