A Mixed Methods Case Study of Evidence-Based Practice in a Knowledge Organization

Evidence-based practice offers a key strategy for closing the gap between research and practice in organizational and management studies. This approach calls for practitioners to apply key critical thinking competencies to gather and use the best available evidence to inform decision making and action (Rousseau, 2006). As a result, efforts to try and develop the evidence-based practice capabilities of practitioners abound, mainly in the form of workshops and university courses offered by leading proponents in the field. Yet, we know little about the impact of these training approaches and whether they transfer to actual differences in practitioners’ behaviors on the job, the aim of any consequential training program (Baldwin, Ford, & Blume, 2017). We also have a limited understanding of how knowledge workers might attempt to implement evidence-based management practices as compared to evidence-based practice in more established areas such as medicine (see Sackett, 2000). This study addresses the lack of understanding about evidence-based practice through a case study of a small knowledge organization using mixed methods. First, an experimental design (n=27) was used to assess whether a set of training modules focused on three core evidence-based practice competencies increases competence in evidence-based practice. In addition, non-experimental designs (n=20- 31) were used to assess how competence, a disposition towards critical thinking (i.e., consistent internal motivation), and opportunities to use evidence-based competence predict application to practice. The results from these quantitative analyses revealed that the training was viewed favorably by most and had a large impact on the competence of trainees. However, neither their competence, critical thinking dispositions, or opportunities to use predicted evidence-based practices. Qualitative semi-structured interviews (n=12) and observations of organizational meetings (n=7) were used to examine how these evidence-based practices, whether from training or elsewhere, are applied and what facilitates or hinders that process. The qualitative data were analyzed based on a grounded theory approach that yielded several key themes. For example, the data revealed that any application of competencies from this training or pre-existing abilities focused almost exclusively on research activities rather than typical practitioner tasks. For nonresearch activities, individuals relied on many different types and sources of evidence, often blending them in inconsistent ways. Participants also tended to communicate important evidence-based terminology inconsistently, and little formal structure guided their approach to presenting information. Patterns of responding to evidence use tended to emphasize low levels of scrutiny or not responding at all, which implicitly reinforced how individuals gathered and presented evidence. The key organizational factors driving these behaviors included organizational and team level cultural norms along with role and task demands. Finally, the reported factors influencing evidence-based practices were consistent with previous work (e.g., Barends et al., 2017) regarding the importance of time constraints and organizational culture. However, the results also illuminated several additional factors that matter when individuals have the prerequisite research backgrounds that overlap with the competencies taught in the evidence-based practice training. These factors include role and task constraints, level of group support, and leadership expectations. The results reveal the importance of understanding and leveraging the entire organizational system (e.g., training and culture) to best support evidence-based practices amongst individual practitioners