Comparison between treatment effects in a randomised controlled trial and an observational study using propensity scores in primary care

Background Although randomised controlled trials (RCTs) are considered 'gold standard' evidence, they are not always feasible or appropriate, and may represent a select population. Observational studies provide a useful alternative to enhance applicability, but results can be biased due to confounding. Aim To explore the utility of propensity scores for causal inference in an observational study. Design and setting Comparison of the effect of amoxicillin on key outcomes in an international RCT and observational study of lower respiratory tract infections. Method Propensity scores were calculated and applied as probability weights in the analyses. The adjusted results were compared with the effects reported in the RCT. Results Groups were well balanced in the RCT but significantly imbalanced in the observational study, with evidence of confounding by indication: patients receiving antibiotics tended to be older and more unwell at baseline consultation. In the trial duration of symptoms (hazard ratio 1.06, 95% CI = 0.96 to 1.18) and symptom severity (-0.07, 95% CI = -0.15 to 0.007) did not differ between groups. Weighting by propensity score in the observational study resulted in very similar estimates of effect: duration of symptoms (hazard ratio 1.06, 95% CI = 0.80 to 1.40) and difference for symptom severity (-0.07, 95% CI = -0.34 to 0.20). Conclusion The observational study, after conditioning on propensity score, echoed the trial results. Provided that detailed information is available on potential sources of confounding, effects of interventions can probably be assessed reasonably well in observational datasets, allowing them to be more directly compared with the results of RCTs