Making sense of diagnostic meta-analysis using carotid artery imaging as a template

The methodology of meta-analysis of diagnostic studies is underdeveloped compared to that of randomised controlled trials. However, summarising evidence from diagnostic studies is important, and interest is growing in diagnostic study meta-analysis.Current recommendations are to use summary receiver operating characteristic (SROC) curve methods, in particular the bivariate or Hierarchical SROC (HSROC) methods, which share many statistical properties. These methods have been little used as they are recent developments requiring statistical expertise.To test these methods, a systematic review on the diagnosis of carotid stenosis by four noninvasive tests was undertaken to provide data for the SROC curve methods. Both the bivariate and HSROC methods failed forall four tests.Further investigation of the behaviour of the SROC curve methods, in particular why they did not work for a large proportion of datasets, was undertaken in a simulation study. This found a failure rate of 50%. Failure of the SROC curve methods was more likely when the individual studies were small and the average sensitivity or specificity was high, and these are characteristics of real studies. The DiagMeta package was developed for use in R, a freely available software package —for use by reviewers who may lack methodological expertise —with guidance on when the SROC curve model fails and alternative analyses.Finally, an empirical comparison was made between individual studies' receiver operating characteristic (ROC) curves and their SROC curve. Even when ameta-analysis can be successfully performed, the resulting SROC curve is difficult to interpret and may not lie close to the individual studies' ROC curves. The SROC curve model is difficult to fit and can limited by the data. Reviewers should therefore try to obtain as much data as possible prior to meta-analysis