A statistical investigation of fraud and misconduct in clinical trials

Research misconduct can arise In any area of research and can discredit the findings. Research misconduct at any level is unacceptable, especially in a clinical trial. Because the results from clinical trials are used to decide whether or not treatments are effective, and affect decisions that may influence treatment choices for large numbers of patients, the prevention and detection of scientific misconduct in clinical trials is particularly important. Chapter 1 outlines some definitions of research misconduct, discusses the underlying motivations behind it, and the overall prevalence of research misconduct beyond that occurring in clinical trials. Different ways to detect and prevent research misconduct are also presented. In addition, an initial insight into the types of scientific misconduct that have been reported as occurring in clinical trials, based on a search of the PubMed database between January 2000 and July 2003 is provided. Thirty-eight published reports were found, but they provide no indication of the relative importance of different types of scientific misconduct in clinical trials. Chapter 2 presents a three-round Delphi survey aimed at achieving consensus among experts in clinical trials on what types of scientific misconduct are most likely to occur, and are most likely to influence the results of a clinical trial. This study identified thirteen forms of scientific misconduct for which there was consensus (>50%) that they would be likely or very likely to distort the results and consensus (>50%) that they would be likely or very likely to occur. Of these, the over-interpretation of 'significant' findings in small trials, selective reporting and inappropriate sub-group analyses were the main themes. To prevent such types of misconduct in clinical trials, the issue of selective reporting of outcomes or sub-group analyses and the opportunistic use of the play of chance (inappropriate sub-group analyses) should be addressed. Full details of the primary and secondary outcomes and sub-group analyses need to be specified clearly in protocols. Any sub-group analyses reported without pre-specification in the protocol would need supporting evidence within the publication for them to be justified. Chapter 3 explores selective reporting and inappropriate sub-group analyses within a cohort of randomised trial protocols approved by the Lancet. It determines the prevalence of selective reporting of primary and secondary outcomes and sub-group analyses in published reports of randomised trials. It also examines how sub-group analyses are described in protocols and how sub-group analyses are reported, and whether they match those specified in the protocol. Of 56 accepted protocols, four non-randomized trials were excluded. For the remaining 52, permission to review them was obtained for 48 (92%). Of those 48 trials, 30 (63%) trials were published. This study identifies some shortcomings in the reporting of the results of primary and secondary outcomes and sub-group analyses. It shows at least one unreported primary, secondary or sub-group analysis in 37%, 87%, and 50% of the trials, respectively. It also shows that the pre-specification and reporting of sub-group analyses are often incomplete and inaccurate. The majority of protocols gave hardly any detail on this matter. There was notable deviation from the protocols in reports in several of the trials. Data fabrication and falsification were judged by the experts in the Delphi survey to be unlikely to occur. However, they can have major effects on the outcomes of clinical trials if it they do occur. A systematic review was conducted in chapter 4, to identify the available statistical techniques that could be used for the detection of data fabrication and falsification. Chapter 5 examines the ability of these statistical techniques to detect data fabrication in data from two randomised controlled trials. In one trial, the possibility of fabricated data had been raised by British Medical Journal (BM]) referees and the data were considered likely to contain fraudulent elements. For comparison, a second trial, about which there were no such concerns, was analysed using the same techniques, and no hint appeared of any unusual or unexpected features was shown. Finally, chapter 6 contains some concluding remarks, a discussion of the strengths and weaknesses of this research and suggestions for future research