Uncertainty in Meta-Analysis: Bridging the Divide Between Ideal and Available Extracted Data

Meta-analysis in the health sciences combines evidence from multiple studies to derive stronger conclusions about the efficacy of treatments. In the process of data extraction from published papers, it is extremely common for the required data to be ambiguous, incomplete or missing. We consider the case of meta-analysis of odds-ratios with unknown number of events and meta-analysis of mean differences with missing standard errors. Existing approaches consist of computing best-estimates for the missing values then feeding them into the meta-analysis as extracted data without accounting for the uncertainty of the computations. These naive approaches lead to over-certain results and potentially inaccurate conclusions. Meta-analysis of odds-ratios assumes binomially distributed numbers of events in each treatment group and requires extracted number of events, which are often not available due to loss to follow-up. Common practice consists of inferring the probability of survival from measurements of the Kaplan Meier survival plot and then using it to infer the number of deaths. We propose the Uncertain Reading-Estimated Events model to construct each study's contribution to the meta-analysis separately using the data available for extraction. In our meta-analysis comparing CABG and PCI for ULMCA stenosis, accounting for the uncertainty results in increased standard deviations of the log-odds as compared to a naive meta-analysis that assumes ideal extracted data, equivalent to a reduction of the overall sample size of 43\% in our example. Simulations show that meta-analysis based on the observed number of deaths lead to biased estimates while our model does not. Meta-analysis of mean differences requires extracted mean differences and their standard errors (SE). However, missing standard errors are pervasive in publications. An algebraic computation to recover the missing SE utilizes the baseline and follow-up standard deviations, and correlations, which are also typically missing. Traditional approaches, that have not been theoretically derived, replace missing SEs with various single-value imputations. We formally derive the Uncertain Standard Error Bayesian model to accommodate multiple patterns of missingness in the standard deviations. In our meta-analysis comparing home monitoring blood pressure to usual care, accounting for the uncertainty results in larger posterior SEs compared to the traditional approaches