Defending psychiatry or defending the trivial effects of therapeutic interventions? A citation content analysis of an influential paper

Aims. Leucht et al. in 2012 described an overview of meta-analyses of the efficacy of medication in psychiatry and general medicine, concluding that psychiatric drugs were not less efficacious than other drugs. Our goal was to explore the dissemination of this highly cited paper, which combined a thought provoking message with a series of caveats.Methods. We conducted a prospectively registered citation content analysis. All papers published before June 1st citing the target paper were independently rated by two investigators. The primary outcome coded dichotomously was whether the citation was used to justify a small or modest effect observed for a given treatment. Secondary outcomes regarded mentioning any caveats when citing the target paper, the point the citation was making (treatment effectiveness in psychiatry closely resembles that in general medicine, others), the type of condition (psychiatric, medical or both), specific disease, treatment category and specific type. We also extracted information about the type of citing paper, financial conflict of interest (COI) declared and any industry support. The primary analysis was descriptive by tabulating the extracted variables, with numbers and percentages where appropriate. Co-authorship networks were constructed to identify possible clusters of citing authors. An exploratory univariate logistic regression was used to explore the relationship between each of a subset of pre-specified secondary outcomes and the primary outcome.Results. We identified 135 records and retrieved and analysed 120. Sixty-three (53%) quoted Leucht et al.'s paper to justify a small or modest effect observed for a given therapy, and 113 (94%) did not mention any caveats. Seventy-two (60%) used the citation to claim that treatment effectiveness in psychiatry closely resembles that in general medicine; 110 (91%) paper were about psychiatric conditions. Forty-one (34%) papers quoted it without pointing towards any specific treatment category, 28 (23%) were about antidepressants, 18 (15%) about antipsychotics. Forty (33%) of the citing papers included data. COIs were reported in 55 papers (46%). Univariate and multivariate regressions showed an association between a quote justifying small or modest effects and the point that treatment effectiveness in psychiatry closely resembles that in general medicine.Conclusions. Our evaluation revealed an overwhelmingly uncritical reception and seemed to indicate that beyond defending psychiatry as a discipline, the paper by Leucht et al. served to lend support and credibility to a therapeutic myth: trivial effects of mental health interventions, most often drugs, are to be expected and therefore accepted. Protocol registration: https://osf.io/9dqat