The external validity of randomized controlled trials: Outpatients and the psychotherapy outcome literature

In order to determine the extent to which published randomized controlled trials (RCTs) of psychotherapy can be generalized to a sample of community outpatients, a method of matching was employed, using information obtained from outpatient charts to inclusion and exclusion criteria from published RCT studies. The use of this procedure in study 1 revealed that 80% of the patients who had diagnoses represented in the RCT literature were judged eligible for at least one published RCT, but that 58% of the patients had primary diagnoses such as adjustment disorder or dysthymia, which were not represented in the existing psychotherapy outcome literature. The most common reasons that patients in this sample did not match with published RCTs for psychotherapy were (1) having a diagnosis that is not represented in the current RCT literature, (2) failing to meet minimum severity or duration criteria, and (3) failing to meet demographic criteria. Study 2 employed the same methodology with a sample of outpatients who were screened for, but not randomized into, an RCT. In this sample, more patients with substance use disorders or social anxiety disorder were not eligible for at least two RCTs. Common reasons that patients did not match with at least two published RCTs for psychotherapy included: (1) patients were in partial remission, (2) failure to meet minimum severity or duration criteria, (3) treatment with antidepressant medication, and (4) the disorder being studied was not primary (mostly for social anxiety patients). Study 3 involved comparing patients who were not randomized into studies for major depressive disorder to those who were randomized on diagnostic and demographic variables. Non-randomized patients were of a comparable level of severity as randomized patients, and tended to have either comparable or significantly lower rates of Axis I and II comorbidity, substance abuse, and recurrence. While more non-randomized patients were non-white and unemployed than randomized patients, these characteristics appeared to be associated with higher rates of comorbidity among the non-white, unemployed patients. The implications of these findings for future research and clinical practice are discussed