Assessing an Assessment: A Case Study of the NSSE ’Experiences with Information Literacy’ Module

This case study explores anomalous results from an administration of the \u27Experiences with Information Literacy\u27 (IL) add-on Topical Module to the National Survey of Student Engagement (NSSE) at Grand Valley State University (Allendale, Michigan) in 2016, and finds that wording of NSSE survey questions can significantly influence results in certain contexts. Overall, summary responses for participating students compared positively to the aggregate means for all participating Large Public institutions in the 2016 NSSE cohort, on both the core NSSE survey and the IL module. However, analysis of local responses to the IL module questions broken out by individual colleges within the university revealed an anomaly. Students in GVSU\u27s College of Engineering and Computing appeared to report very low engagement on nearly all of the items in the IL module; further disaggregated into separate programs comprising the college, data appeared to perhaps indicate that Engineering students’ educational experience with respect to information literacy learning at GVSU is qualitatively different from that of their peers in other academic and professional disciplines, even within their own college, which also includes Computer Science and Occupational Safety and Health. In 2018 Senior GVSU Engineering majors received a modified NSSE-IL survey (with permission obtained from NSSE), to explore whether Senior GVSU Engineering majors may be graduating with lesser information literacy learning preparation than other GVSU graduates. Results suggest that revising NSSE-IL framing questions does result in some significant changes in rates of certain responses, some tending in a positive direction toward the institutional mean, others tending negatively away from it. We conclude that NSSE-IL in 2016 has in fact allowed us to observe an anomaly, that Seniors in one specific program do not share a perception of information literacy experiences in common with their peers in other programs at the same institution; this, in spite of wording in the survey instrument that includes built-in assumptions that, taken at face value, could have led to an inaccurate or misleading profile of GVSU Engineering graduates\u27 experience