Results on inter-rater-reliability.

BackgroundHealth information is a prerequisite for informed choices–decisions, made by individuals about their own health based on knowledge and in congruence with own preferences. Criteria for development, content and design have been defined in a corresponding guideline. However, no instruments exist that provide reasonably operationalised measurement items. Therefore, we drafted the checklist, MAPPinfo, addressing the existing criteria with 19 items.ObjectivesThe current study aimed to validate MAPPinfo.MethodsFive substudies were conducted subsequently at the Martin Luther University Halle-Wittenberg, Germany and the Medical University of Graz, Austria: (1) to determine content validity through expert reviews of the first draft, (2) to determine feasibility using ‘think aloud’ in piloting with untrained users, (3) to determine inter-rater reliability and criterion validity through a pretest on 50 health information materials, (4) to determine construct validity using 50 developers’ self-declarations about development methods as a reference standard, (5) to determine divergent validity in comparison with the Ensuring Quality Information for Patients (EQIP) (expanded) Scale. The analyses used were qualitative methods and correlation-based methods for determining both inter-rater reliability and validity.ResultsThe instrument was considered by experts to operationalise the existing guidelines convincingly. Health and nursing science students found it easy to understand and use. It also had good interrater reliability (mean of T coefficients = .79) and provided a very good estimate of the reference standard (Spearman’s rho = .89), implying sound construct validity. Finally, comparison with the EQIP instrument revealed important and distinct areas of similarities and differences.ConclusionsThe new instrument is ready for use as a screening instrument without the need for training. According to its underpinning concept the instrument exclusively comprises items which are justified by either ethics or research evidence, implying negligence of not yet evidence based, however, potentially important criteria. Further research is needed to complete the body of evidence-based criteria, aiming at an extension of the guideline and MAPPinfo.Trial registration numberAsPredicted22546; date of registration: 24 July 2019.