Classifying Student Performance as a Method for Setting Achievement Levels for NAEP Writing

One way standard setting methods can be distinguished is by whether they are test-centered or examinee-centered (Jaeger, 1989; Kane, 1998). In test-centered methods judges decide on a level of performance that is considered just adequate for the standard to be met on each item or task in the test. The item-level judgments are combined to obtain a set of performance level cutpoints for the test as a whole. In examinee-centered methods judges categorize examinees directly into performance levels using deflnitions of adequate performance for each level and information about the level of achievement of each examinee. Test-centered methods have been used for setting NAEP standards. This paper examines an examinee-centered standard setting method (Booklet Classiflcation) as a possible standard setting method for the 1998 NAEP Writing assessment. The study described in this paper is the second NAEP 1998 Writing fleld trial for achievement levels-setting. There are three NAEP achievement levels: 1) basic, 2) proflcient, and 3) advanced. Achievement Level Descriptions (ALD) describe what students should know and be able to do at the basic, proflcient and advanced achievement levels. Achievement that doe