A psychometric analysis of an alternate assessment for students with multiple disabilities

Although students with severe and multiple disabilities have participated in state-level alternate assessments for years, research has shown that their educational achievement has not been measured in a psychometrically sound way (Kettler et al., 2010; Elliott, Compton, & Roach, 2007). Therefore, schools cannot meaningfully evaluate these students’ progress. Professionals within PG Chambers, a charter school in New Jersey serving students with severe disabilities, recognized the need for an evaluation tool that could quantify the difference that their school program makes in the lives of their students. To assess both therapeutic and academic progress for students with multiple disabilities, in 2008 a transdisciplinary team created the PG Chambers School Outcomes Measurement Tool (PGS-OMT). By gaining insight into three sets of psychometric properties of the PGS-OMT, this study aimed to determine how meaningfully students at PG Chambers are being evaluated. First, the reliability of the PGS-OMT was evaluated through analyses of internal consistency and cross-informant agreement. Reliability analyses using Cronbach’s alpha indicated that scores demonstrated excellent internal consistency, and Pearson correlations indicated that scores demonstrated high cross-informant agreement. Second, the internal structure of the PGS-OMT was assessed through correlations among the subscales, confirmatory factor analysis (CFA), and exploratory factor analysis (EFA). The internal structure analyses were not conclusive. CFA yielded results that were difficult to interpret because of high factor loadings for both the uncorrelated and the correlated three-factor model coupled with poor fit indices. EFA yielded findings supportive of the three-factor model rejected by the CFA. Strengths and weaknesses of the one-, two-, and three-factor models were therefore discussed. Third, change in scores from 2013 to 2014 of the PGS-OMT was assessed using descriptive statistics, paired samples t-tests, and Cohen’s d. Change in scores analyses yielded significant t-tests, indicating that students at PG Chambers are generally improving from year to year on some domains measured by the PGS-OMT.Psy.D.Includes bibliographical referencesby Samantha Schulma