Programmatic actions upon assessment results: A case study of breadth and depth in higher education

Student learning outcomes assessment has become a regular institutional activity, in which faculty members investigate whether students have achieved the expected knowledge, skills, and dispositions at the course, program, and/or institutional level. The main purpose of learning assessment should be for education improvement. It is common for every higher education institution to collect assessment reports periodically. The analysis of the reports often reveals the scope and extent that courses/programs have used assessment results. Often, assessment reports are mainly used to serve accreditation purposes. The driving question of this research is: how to best utilize the assessment reports themselves to promote programmatic change for educational improvement. Three operationalized evaluation questions were developed to answer this question: 1. What are the breadth and depth of use-of-assessment-results in academic degree programs for educational improvement? 2. What are the factors associated with excellent use-of-results? 3. How can higher education institutions use the assessment reports to promote programmatic change for educational improvement? The research was conducted at a large research university using content analysis of program assessment reports collected from 238 academic programs. We included reported from 164 programs who claimed use-of-results. The breadth of the use-of-results was investigated through content categorization analysis, or theme analysis, of the assessment reports. The results show major categories of use as: assessment tools and processes, course curriculum, program curriculum and policy, resource and personnel, student support, and celebration. Sub-categories under each category were further identified. For example, assessment tools and processes include creating/modifying student learning outcomes, creating/modifying curriculum maps, creating/modifying rubrics, and etc. The depth of use was analyzed through both holistic expert judgment and judgment on the level faculty collaboration in programmatic decisions. About 20% of the reports (n=33) were coded by the second rater. Discrepancies in coding were resolved through consensus. Finally, the presentation recommends ways to use and communicate the findings to campus in order to motivate more programs to use assessment results for improving learning.This research used content analysis to analyze student learning outcomes assessment reports from 164 academic degree programs at a large research-oriented university. The findings present the breadth of use-of-assessment-results through analyzing major types of use (e.g., program curriculum content and policy) and how each type has been manifested (e.g., adding pre-requisites) most frequently. The depth of use was judged by faculty involvement in programmatic decisions and the expert holistic judgment. The presenter recommends ways to use this type of meta-synthesis to promote more programmatic actions for improving learning on campus