A PART Tragedy: The Case of Upward Bound Correcting for Study Error in the Random Assignment 1992-2004 National Evaluation of Upward Bound

serving as the project director for national evaluations of Student Support Services and Talent Search, and the UB performance reporting contracts. She also served as survey director for the Third Follow- up and initiation of the Fourth Follow-up data collections for the study that is the subject of this article. Abstract: The paper takes a critical look at study error relative to findings from the National Evaluation of Upward Bound (UB) and explores policy and methodological lessons learned. The nationally representative random assignment study followed a multi-grade cohort from 1992-93 to 2003-04. Following published reports of lack of overall positive effects, OMB gave UB an ineffectual Performance Assessment Rating Tool (PART) rating and Bush Administration budget recommendations in FY05 and FY06 called for zero funding. Efforts were also made to reform the program, by requiring projects to serve more academically at-risk students. Major technical issues discussed include: 1) extreme unequal weighting; 2) treatment-control group non-equivalency; 3) survey non-response bias; 4) lack of standardization for expected high school graduation year (EHSGY) in outcome measurement; and 5) service substitution and dropout issues. Our major finding is that when administrative records are used to supplement survey data and outcomes standardized by EHSGY, contrary to previously published reports, the UB program demonstrated statistically significant positive effects on the major goals of the program; postsecondary entrance, application for financial aid; and attainment of postsecondar