Evaluating the pre-operational and operational test instruments and testing strategies for the Gearing Up for Safety: Production Agricultural Safety Training for Youth curriculum

Current youth training and certification efforts, under the Agriculture Hazardous Occupations Order (AgHOs) regulations set forth by the Fair Labor Standards Act (FLSA), have not been standardized and in some cases have been determined to not meet even basic requirements of the order. This is especially true for hands-on evaluation components of the training intended to evaluate operator readiness to safely operate agricultural tractors and equipment. Variability in training and certification methods potentially leaves some youth, who are unprepared, at risk when encountering hazards while employed to operate allowable agricultural equipment. The goal of this project was to evaluate Purdue University’s Gearing Up for Safety: Production Agriculture Safety Training for Youth (Gearing Up for Safety) curriculum’s assessment components designed to test tractor pre-operational and operational abilities of youth. Through a content validity study, the Pre-Operational and Operational Exams were determined to be reliable by a group of subject matter experts. Passing scores were also refined and finalized for each exam instrument: 70% for the Written Exam, 85% for the Pre-Operational Exam, and less than 15 infractions for the Operational Exam. Completion and critical evaluation of the Pre-Operational and Operational testing components complemented the previously validated Written Examination. In addition to gathering validity evidence, the Pre-Operational and Operational Exams were determined, through very high correlations of the data, to provide consistent results with multiple examiners. Instructional resources were also created to assist program leaders when administering the Pre-Operational and Operational Exams. The result of administering the Written Exam utilizing a pre-test/post-test method showed that there was a knowledge gain of 18.6% (13 questions). While there was an average increase of 18.5% in the mean score, the percentage of participants passing increased from 5.9% to 57.6%. Of those participants passing the Written Exam and completing the Pre-Operational Exam, a total of 38.5% passed. A total of 98.7% completing the Operational Exam passed, which indicated those participants lacking required knowledge and skills had been identified and removed during one of the first two stages of testing. The data further showed that 17.5% of participants successfully completed all three stages of testing. These findings suggest that the overwhelming majority of youth enrolled in the current AgHOs certification process lack the knowledge or skills to be employed in agricultural production occupations