Client Report of Session Content in an Effectiveness Trial: In Search of Efficient Fidelity Measurement

The Modular Approach to Treatment for Children with Anxiety, Depression and Conduct problems (MATCH-ADC), a flexible evidence-based intervention, was recently found to be as effective as standard protocols in treating youth with multiple disorders in community mental health settings. A challenge to its implementation is that it requires ongoing, real-time feedback to plan treatment practices. Fidelity measurement, a method of monitoring treatment practices, may be an ideal mechanism for supplying this feedback. However, a majority of existing fidelity measures utilize labor-intensive methods that are impractical for community settings. Alternative methods, such as therapist- and client-report measures, may provide a more appropriate fit and research has supported their feasibility and validity. The present study focused on developing a qualitative client-report measure, the Service Review Measure (SRM), and quantitative coding system for the MATCH intervention. Primary aims were to evaluate the SRM's utility in providing reliable and valid feedback on fidelity. Additional aims were to examine feasibility of the phone call strategy, agreement between caregivers and youth and agreement over time. Fifty-two families with youth ages 7 to 15 participating in an effectiveness trial comparing MATCH-ADTC to usual care (UC) in three community clinics were administered the SRM a maximum of four times over the course of treatment. Their responses were transcribed and then coded according to various domains of session content and reference to specific practices. Interrater reliability for the coding system was found to be acceptable to excellent. Results indicated that clients were able to report conceptually meaningful information and a moderate amount of detail on session domains on average. Clients also reported a similar amount of specific practices as therapists per session but the specific practices endorsed by clients were not highly correlated with, and did not account for, much of the variance in therapist report. Caregiver and youth report on specific practices were also not highly correlated but results showed that, in line with the literature, an externalizing behavior target problem area was significantly associated with an increase in caregiver extensiveness ratings. The same was true for internalizing behavior problem areas and youth report. Further results supported the feasibility of the phone call strategy according to standards set forth in the study but suggest that phone calls are not the most efficient strategy for obtaining client information. Agreement between client and therapist report did not improve over time. Overall, results suggest that there might be differences in how clients and therapists view and choose to report on session practices. In this way, clients may offer a unique perspective that is important in guiding feedback. Future work will compare both client and therapist report to coded tapes in order to determine which report is more accurate