Analysis of Reliability and Validity in Refereeing of Kumite in Karate-Do

Dissertation presented towards Master of Sports Science Degree with specialisation in Sports TrainingMaster's Supervisor: Félix Luís de Lima e Cunha Hopffer Romero PhDAbstract An investigation centered on referees has been relatively scarce or non-existent, particularly in regards to refereeing Kumite in Karate. The present study’s objective was to characterize Karate referees’ performance in a Kumite match along with its validity, as well as the intra and inter-observer reliability. This current investigation focused on observations made by Karate referees belonging to the FPKS and entailed a Karate match that lasted two minutes. Based on that match a eight minute video was put together, where it can be seen from the perspective and position of both the referee and each one of the judges. The data was collected during two different periods of time. The first data collected was to analyse the inter-observer’s reliability and the validity. The second data collection took place within a minimum sixty day interval, in order to verify the intra-observer’s reliability. In order to analyse the validity, a group of experts was commissioned with the purpose of putting together criteria of measure. A sample part of the video was observed individually by the referees, in a closed environment setting and documented with the use of the Match Vision Studio software. The Bellack Index was used to assess the intra and inter-observer’s reliability and validity, therefore accepting a result equal or superior to eighty percent as the guarantor of reliability (Aranha, 2007, Siedentop, 1983). The chi-squared test was used to verify if there were any significant differences in the validity between national and regional referees, as well as, in accordance with the position of the observation (referee or judge). The reliability results were (78,7%) and (77,2%) for the intra-observer and the inter-observer reliability, respectively. The results are in close proximity to each other and one can conclude that the variability between the various referees is not greater than the variations that occur in the same referee at two different periods of time. The validity is low (72,2%), seemingly there is a difference in the assessment criteria between the experts and the referees. This should be improved with the aid of specialised training, in order to better the uniformity of decision criteria. In both the intra and inter-observer’s reliability, no significant differences were found between the national and regional referees. A significant difference of (3,4%, Sig 0,004) was found between the national and regional referees regarding validity, and being even higher for the national referees. The difference between referees and judges is significant when it comes to the intra-observer’s reliability which was (9,9% Sig 0,002) and (9,8%, Sig 0,004) for validity, with the judges being even higher. A conclusion can be made that validity was always lower than the intra and inter-observer’s reliability, which denotes a divergence in the assessment criteria between referees and experts. The lack of the intra-observer’s reliability seems to be a major constraining factor for a better performance