Reducing the subjective impact in maritime simulator assessment A performance assessment tool for maritime pilotage operations

Maritime navigation involves the process of monitoring and controlling the movement of a ship from one position to another. When the ship is approaching or leaving a port, a local navigational expert, the pilot, is often provided for assisting the bridge team to safely and efficiently navigate the littoral waters. However, statistics concerning maritime accidents continues to associate human errors with accidents, which advocates a scrutiny of maritime education and training (MET) for pilotage operations. The pilotage operation is a unique phase of the voyage in which the bridge team has to rely on an external expert with whom they have no prior working experience with. Also, the pilot-bridge team often has to perform immediately, and the consequences of not performing can be calamitous. These aspects suggest a need for dedicated research on how to ensure that bridge teams have the necessary competencies for carrying out safe and efficient pilotage operations. Training and assessment are quintessential instruments for this mission. MET facilities increasingly rely on full-scale simulators for their benefits in training for complex operations. Meanwhile, the assessment of training performance is often based on subjective criteria, which has implications on the reliability and validity of the assessment. Consequently, lacking a reliable and valid assessment method could have repercussions on the educational quality, as well as subsequent developments to the MET facilities’ training and education programs. The aim of this research is to understand and to advance performance assessment related to the use of full-scale maritime simulators in MET, with a main objective to reduce the subjective impact that can be present in the performance assessment of pilotage operations. This thesis presents a computer-assisted assessment tool based on a structural probabilistic network, in which the assessment criteria are weighted by using an analytical hierarchical process. This tool is designed for flexibility, so it can easily be employed across a multitude of pilotage scenarios, and to be less subjective so that it can present more reliable and valid performance information to its stakeholders. The results show that the presented assessment tool can be used for higher reliability in the assessment of technical performance, but that more research is required to assess teamwork performance reliably. The tool’s content validity is considered adequate, and that studies over time are required to assess its criterion validity effectively. Regardless, the tool is deemed opportune for generating precise and accurate assessment of training performance and could serve as a steppingstone to objective assessment. Accurate and precise assessment of training performance are imperative for stakeholders that make executive decisions concerning the development of training programs, competency mappings of the workforce, as well as for the trainee to know his or her strengths and weaknesses in the operation. Providing executive stakeholders with information for making decisions that are based on objective performance assessment data could serve as a piece of the puzzle in the mission to reduce human errors in maritime shipping. Keywords: Maritime; Performance Assessment; Training; Pilotage Operations; Human Factors; Full-scale Simulators; Maritime Education and Trainin