ADVANCING THE EVALUATION METHODOLOGY FOR COMMUNITY-BASED INJURY PREVENTION PROGRAMS: PERSPECTIVES FROM A CASE STUDY EVALUATION

Background: The traditional public health approach to injury prevention and control is founded on a causal model of injury that focuses on individual-level risk factors relevant to the exchange of energy that causes anatomical injury. Contemporary epidemiology recognises the role of macro-level physical and social determinants in injury aetiology and therefore the ecological approach, which examines the population level effects on health outcomes, has become more frequently employed. The ecological approach underpins the community-based model of injury prevention. Despite implementation of this model on a global scale, few evaluations demonstrate the effectiveness of the community-based model to reduce injury outcomes. Existing evaluations are limited by methodological flaws including inadequate evaluation timeframes, poor selection of community controls, use of poor quality data sources and failure to match the evaluation approach with the ecological nature of the community-based model. Current evaluation methods do not adequately account for the effect of social variables upon injury rates. Aim: The aim of the thesis is to advance evaluation methodology for community-based injury prevention programs by: designing and implementing a case study evaluation program consistent with current best practice methods for community-based injury prevention programs; using the quantitative data from the case study evaluation to illustrate threats to the validity of results obtained by this methodology; and developing an improved methodology for application to future evaluations of community-based injury prevention programs. Methods: A quasi-experimental study which measured changes in process, impact and outcome evaluation components in two matched intervention and control community pairs was conducted between May 2003 and September 2005. Program implementation activities were recorded by the project officers in electronic process logs. Impact evaluation included assessment of community capacity before and after program implementation using a Community Capacity Index, and measurement of pre- and post-implementation prevalences of safety behaviours and injury risk factors using self-report surveys. Outcome evaluation included injury data from emergency department and hospital admissions datasets for thirteen years pre-implementation and eighteen months after program implementation commenced. Effectiveness of the case study programs was assessed by comparing predicted and observed injury rates. A generalised estimating equation model of childhood injury rates was created using state wide data to control for changes in social variables within the two intervention communities. Results: The effect of the prevention program on the intervention communities was minimal. Little change in process activities was demonstrated. Community capacity improved slightly in both communities; however changes to safety behaviour and injury risk factor prevalence were limited. Many of the significant changes represented increased injury risk. Changes to outcome measures were also limited. Observed childhood injury rates were within predicted ranges for Intervention and Control Community One and Intervention Community Two, but increased beyond the predicted range in Control Community Two. The GEE model indicated the intervention was associated with statistically non-significant decrease of 0.07 injuries/10,000 0-4 year olds (95%CI: -0.24, 0.10) in logarithmically transformed injury rates (p=0.40).The GEE model found a significant relationship between childhood injury rates and first (p<0.001) and second order time components (p<0.001). Rurality was significantly associated with childhood injury rates, where childhood injury rates increased by 0.06 injuries/10,000 0-4 year with each increasing unit of remoteness (p<0.001). The model found an increase of 2.69 injuries/10, 0000 0-4 year olds as the proportion of married people within the region increased (p<0.001) and a decrease of 1.84 injuries/10,000 0-4 year olds as the proportion of Indigenous residents increased (p<0.001). There was also a significant association between the proportion of Indigenous residents, time and childhood injuries (p<0.002). When controlling for social variables, the intervention was associated with a decrease of 0.09 injuries/10,000 0-4 year olds (95%CI: -0.29, 0.11) in logarithmically transformed injury rates, however this decrease was not significant (p=0.36). Conclusion: This thesis has explored the need to consider the effect of social level variables on rates of childhood injury. The lack of identification, data collection and analysis of social variables when evaluating community-based injury prevention programs may explain the lack of observed effectiveness of community-based programs. The current best practice methodology for evaluating community-based injury prevention programs can be improved by ensuring the effect of social variables are accommodated with the analytic model of the evaluation