Evaluation of quality indicators for Dutch community pharmacies using a comprehensive assessment framework

BACKGROUND: In 2008, the Dutch Health Care Transparency Programme (Zichtbare Zorg) was set up to develop and apply quality indicators (QIs) for health care. These QIs serve a range of purposes and can be categorized into those for internal use--for meeting quality standards and to continuously measure improvement (formative)--and external use--to enable patients and health insurance companies to distinguish between health care providers (summative). In order to assess the validity of QIs, a comprehensive Indicator Assessment Framework (IAF) was developed. This framework specifies the following criteria for validation: content validity, absence of selection bias, absence of measurement bias, and statistical reliability. Because of the intended summative use, the IAF was used for structural assessment of the QIs set for Dutch community pharmacists. OBJECTIVE: To assess the validity of the current set of 52 QIs for community pharmacies using the IAF. METHODS: An expert panel applied the IAF criteria to the set of QIs collected in 1,807 Dutch community pharmacies on their performance in 2011. The QIs were judged as meeting, partly meeting, or not meeting the requirements regarding these criteria. The judgments were evaluated for QI type (structure, process, or outcome) and for predefined domains. RESULTS: Thirteen QIs (25%) were judged as meeting the requirements for all criteria. Among them were 12 structure indicators and 1 process indicator. For process indicators, the criterion for measurement bias poorly met the requirements, and content validity was unsatisfactory for outcome indicators. The 13 overall valid QIs covered 6 out of 10 predefined domains: continuity of care, clinical risk management, compounding, dispensing of medication, management, and quality management. CONCLUSIONS: When subjecting the QI set for community pharmacies to the requirements of the IAF, only a quarter of the QIs met all requirements. To increase the number of valid process and outcome indicators, meaningful aspects for the outcome of pharmaceutical care have to be defined, and uniform measurement of relevant processes has to be implemented