Assessing the Relationship between SafeCare Fidelity and Competence Measures

As more evidence-based programs are implemented in community settings, there is a strong need to ensure those models are implemented with integrity. Implementation of programs should be evaluated for fidelity, the degree of adherence to treatment protocols, and competence, the level of skill in implementation (Schoenwald et al., 2011). The purpose of this study was to review audio recordings of SafeCare home visiting sessions to discover the relationship between the measures of fidelity and competence. Six coders were assigned 209 SafeCare home visiting audiotapes to be coded for fidelity and competence. A sample of audios were double coded to evaluate fidelity and competence scores for inter-rater reliability. Fidelity and competence items were classified into process and content categories, forming the six main variables of process fidelity, content fidelity, total fidelity, process competence, content competence, and total competence. Total fidelity correlated with total competence at a level of .615, with process fidelity and process competence correlating at a much lower level than content items. The total correlation level can be interpreted as that fidelity and competence are strongly related measures, but are not identical constructs. The goal for SafeCare coders would be to continue refining competence definitions and attempting to remove the subjective nature from the competence coding process. With these two efforts, competence reliability should increase to an acceptable level. Given the main fidelity and competence correlation level, it is advisable for SafeCare coders to continue to code both fidelity and competence to avoid missing valuable components of the session. Additional research may be needed once the competence scale becomes better established