Inter-rater and intra-rater agreement in causality assessment of adverse drug reactions: a comparative study of WHO-UMC versus Naranjo scale

Background: The causality assessment system proposed by the WHO collaborating centre for international drug monitoring, the Uppsala monitoring centre (WHO-UMC) and the Naranjo probability scale are the generally accepted and most widely used methods for causality assessment. Both these scales are structured, transparent, consistent, and easy to apply with distinct advantages. The PvPI recommends use of WHO-UMC scale while many clinicians prefer Naranjo scale for its simplicity. As both these scales are used very widely in practise, it is important to study the level of agreement among them, to ensure proper interpretation of the causality assessment of the drugs. Objective was to study the inter-rater and intra-rater agreement in causality assessment of adverse drug reactions between WHO-UMC and Naranjo scale.Methods: A total of 200 cases were analyzed by three raters. Each ADR reporting form was analyzed by the raters independently using both the Naranjo scale and the WHO-UMC scale. The inter-rater and Intra-rater agreement of all the three raters was analyzed using kappa statistics.Results: The most common category of causality assessment was ‘possible’ which was around 73%, while the probable, definite and unlikely accounted for 23%, 3% and 1% respectively. The inter-rater agreement for the various categories of causality assessment when using Naranjo scale ranged from “very good to excellent” (Kappa value 0.95) while the same ranged from “good to very good” when the WHO-UMC scale was used (Kappa value 0.89) while the Intra-rater agreement for the three raters ranged from good to very good. The mean time (in minutes) taken for assessing the suspected adverse drug an ADR was 8. 26±2.53 for WHO-UMC scale while it was 14.18±3.44 when Naranjo scale was used.Conclusions: Both the WHO-UMC scale and Naranjo scale are reliable and valid tools for causality assessment but the Inter-rater agreement was slightly better with Naranjo as compared to WHO-UMC scale