A longitudinal mixed method approach for assessing implementation context and process factors: Comparison of three sites from a Housing First implementation strategy pilot

Background: Implementation science’s focus on establishing implementation strategy effectiveness has overshadowed the need to understand differential performance of such strategies under various conditions. Methods allowing for assessment between implementation context and process can help address this gap. This article provides a detailed description of a mixed method procedure for assessing factors related to the implementation context and process intersection, which was developed as part of the pilot study of the Housing First Technical Assistance and Training (HFTAT) program, a multifaceted strategy designed to support Housing First model implementation. Methods: The HFTAT was pilot tested among a sample of three organizations. Our mixed method approach combines two tools often used in implementation research—the Stages of Implementation Completion (SIC) and the Consolidated Framework for Implementation Research—in a novel way. Several stages to analysis were completed, starting with a separate analysis of data pertaining to each measure and then two levels of mixed method analysis. Results: The approach provided a better understanding of the issues that impacted the implementation guided by the HFTAT, suggesting (1) individual determinants seemed to have a bigger impact based on the number of SIC phases they affected, (2) implementation context and process were connected through climate-related factors in the inner setting that made the sites more or less responsive to addressing identified barriers, and (3) there is a need to better assess context factors to identify areas where implementation drivers should be better targeted to facilitate change, and this is supported by prior research. Conclusions: Understanding the underlying factors impacting a setting’s performance related to a specific implementation strategy has potential to improve decision-making and optimize future implementation efforts. The approach likely be as successful combining the SIC with other determinant frameworks and should be utilized at the onset of an implementation project to maximize its usefulness. Plain language summary The field of implementation science needs better approaches for understanding how context (i.e., constraints and opportunities in the implementation setting) and process (i.e., the actions and steps taken during implementation) interact over the course of implementation. Such information would be very useful for understanding the success or failure of specific implementation strategies, which are specific techniques used to increase the adoption, implementation, and sustainability of a specific intervention. This article explains a method to assess the intersection of implementation context and implementation process that we developed to better understand issues that could help explain the effectiveness of an implementation strategy for an intervention for housing people who are experiencing chronic homelessness and who have both a diagnosed substance use disorder and serious mental illness. The method combines two popular implementation tools, the Stages of Implementation Completion and the Consolidated Framework for Implementation Research. Using this method resulted in a better understanding of differences in implementation performance at each of the organizations and how we might improve the implementation strategy. This understanding was better than what we had learned from other approaches we had used before this. We provide some suggestions for how the method can be strengthened for use in other studies