Refining questionnaire items in a Patient Reported Experience Measure (PREM) designed for users of NHS ambulance services: question appraisal study

Background Service users’ experience of ambulance services is important as a measure of quality in addition to effectiveness and safety of care. This doctoral study focuses on the development of a Patient Reported Experience Measure (PREM), a questionnaire used to assess people’s experiences of health care provision. The draft PREM had been piloted with a sample of patients to explore their views of the content and format of the questionnaire. In this next stage we aimed to use experts (including ambulance care providers and academics) to evaluate the quality of the PREM. Methods This evaluation involved the use of the Question Appraisal System (QAS-99), originally developed to assist questionnaire designers identify problems with questionnaire items. Participants’ were asked to review 20 items from a draft PREM of 60 items, completing a QAS-99 coding form for each questionnaire item they were assigned to evaluate. The form directed their attention to the type of flaws that may be found, for example, problems with question instructions, clarity of wording, potential for bias and response options. Results Ten participants completed the study. Overall refinements were suggested for 46 of 60 questionnaire items reviewed. Amendments suggested included wording of questionnaire items to improve responders’ understanding and interpretation, appropriateness of response options, and decisions on inclusion/exclusion, of specific questionnaire items. The PREM was revised in accordance with these recommendations. Conclusion The QAS-99 approach to questionnaire evaluation facilitated refinement of the PREM. The final stage of this doctorate will involve further testing of the PREM using cognitive interviews with patients that have recently used the ambulance service to investigate how they interpret and respond to the refined version of the tool