Authentic assessment and its impact on seafarer students’ academic achievement : a comparative analysis with traditional assessment

The Standards of Training, Certification and Watchkeeping (STCW) Convention set global, minimum standards of competence for seafarers in 1978 by introducing the STCW Code (STCW’78). Through amendments in 1995 and 2010, the convention intended to improve the knowledge‐based training mandate established in STCW’78 by making it outcome‐based. With STCW moving seafarer training towards outcomes-based education (OBE), emphasis shifted to assessment practices that will allow seafarer students to demonstrate their ability to perform workplace tasks at standards described in the revised and current STCW’95 Code (including 2010 amendments). Seafarer education and training (SET) institutes working under the directives of the national maritime regulator [e.g. Australian Maritime Safety Authority (AMSA) in Australia] are responsible for ensuring that the adopted assessment methods, as promoted and recommended by the STCW Code, not only assure attainment of STCW outcomes but produce competent graduates that meet the expectations of the core stakeholders such as regulators and employers. However, a critical review of specific excerpts from the STCW Code was used in this research study to show that the Code largely fails to provide a ‘standard’ that can assure assessment of seafarers to one of the most critical outcomes: the performance expected at a level of work in the industry. Moreover, a review of past research in the area of seafarer education conducted for this study, showed that the traditional assessments that required seafarer students to focus on rote learning and construction of responses devoid of real‐world contexts (e.g. oral examinations, written assignments, and multiple‐choice questions) disengaged them from learning. Memorising information is a lower‐order cognitive ability, failure in which led to errors in assessment tasks resulting in low academic achievement for students. Hence, this research proposed that authentic assessments, requiring students to construct responses based on the assimilation, integration, and critical analysis of information presented in real‐world contexts will result in higher academic achievement. Using the characteristics recommended by the commonly cited authors, this study redefined the concept of authentic assessment which established the theoretical framework for this study. However, authentic assessment can capture essential aspects of workplace tasks and result in consistency of student performance in different contexts only if they are valid and reliable. Rubrics as assessment tools are known to increase validity and reliability of assessments, but it can do so only if different aspects of its own validity and reliability have been addressed. An extensive literature review in the area of authentic assessment, conducted as part of this research study, uncovered an absence of academic investigation and empirical study on the different aspects of validity and reliability of authentic assessment, when implemented with as well as without the use of assessment rubrics. In this regard, a conceptual and practical authentic assessment framework for seafarer education and training (AAFSET) that promotes a holistic approach and provides greater assurances of validity and reliability throughout all stages of assessment within seafarer programs was developed in this research study. The findings of the literature review also revealed that there existed an even greater absence of global research on authentic assessment in the area of seafarer training. Hence, an empirical contribution of this study was through the investigation of the difference in seafarer students’ academic achievement (measured through scores obtained in assessment) in authentic assessment as compared with traditional assessment. Two separate and independent student groups as the ‘control’ and ‘treatment’ group were used for a selected unit of learning delivered at the Australian Maritime College (AMC) within the Bachelor of Nautical Science degree program. Since, some past researchers defined traditional assessments as a single‐occasion assessment implemented at the end of the learning period, this project implemented the assessment in a summative format as opposed to authentic assessments implemented formatively. Analysis of student scores revealed that the authentically assessed students were guided towards significantly higher academic achievement. A further investigation using the students undergoing authentic assessment, included measuring their perceptions of authenticity for factors of assessment (task, criteria, etc.) and correlating to their scores in the associated task. Stage 1 focused on deriving the factors conceptually using the definition of authentic assessment, based on which a perception survey questionnaire was designed. Following the collection of student responses through the survey, a correlational analysis was conducted between student perceptions and their scores. Stage 2 extracted new factors through a factor analysis. Using the survey data (but for the new factors), an additional correlational analysis was conducted to confirm findings. Both stages of investigation found that the factor of transparency of criteria was a significant predictor of the students’ academic achievement. Future research will investigate seafarer students’ perceptions through the use of qualitative methodologies (e.g. interviews and focus groups) to gain an in‐depth knowledge of other factors that may enhance authenticity of assessments. Project findings identified vital challenges for the implementation of authentic assessment and uncovered significant factors of assessment which, if included, in the design of the assessment will guide authentically assessed students towards higher academic achievement