A bias-adjusted evidence synthesis of RCT and observational data: the case of total hip replacement

Evaluation of clinical effectiveness of medical devices differs in some aspects from the evaluation of pharmaceuticals. One of the main challenges identified is the lack of robust evidence and a need for combining experimental and observational studies (OS) in quantitative evidence synthesis accounting for internal and external biases. In this project, we addressed this question using the case study of total hip replacement, comparing the risk of revision of cemented and uncemented implant fixation methods. We performed an elicitation exercise to determine the strength of belief of methodological and clinical experts regarding the size of internal and external bias affecting treatment effect estimates. These bias-adjusted treatment effects fed into a generalized evidence synthesis. Comparing several ways of the evidence synthesis showed that the pooled result strongly depends on inclusion vs. exclusion of observational data and on using bias-adjusted estimates or not. Overall, this exercise showed that including OS data in a metaanalysis can increase the evidence body, and therefore, strengthen the evidence. However, to achieve sufficient internal validity, OS data should be either sufficiently controlled for bias in the design and primary analysis, or bias adjustment should be applied to study results to obtain better internal and external validity