1 Running Head: VALIDITY OF ASSESSMENT SCORES FOR ELLS Status 2007: Inspecting the Validity of Large-Scale Assessment Score Inferences for ELLs and Others under More Optimal Testing Conditions—Does it Measure Up?

The goal of the current study was to examine the influence of providing more optimal testing conditions and evaluate the effect this has the validity of the score inferences across ELL students with different needs, strengths, and levels of language proficiency. It was expected that the validity of the score inferences would be similar for 3rd and 5th grade ELL students with different needs and exited and native English speakers who acted as control groups. Multiple choice and constructed response mathematics data from a large-scale data collection were analyzed relative to data from a criterion measure developed for the study and other ancillary information obtained during the project. Results indicated the validity data from multiple choice results for ELLs were generally very poor compared to the control groups, but that validity data from constructed response scores were more promising, especially for beginning students in both grades and advanced students in grade 5. Additional analyses indicated significantly higher misclassification rates of test score data for lower English proficient ELL students as compared to control groups when looking at students classified as knowing some mathematics on the criterion measure. This study raises many questions about the validity of inferences drawn from large-scale assessments for students with lower English proficiency. It also calls into question the effectiveness of measuring content knowledge with some item types for students at various levels of English language proficiency, and suggests that item type may interact with grade level of the test takers