Validity in times of measurement: on the epistemic validity of test validity in psychotherapyresearch

Background and introduction: In psychotherapeutic research on Evidence-based Treatments (EBTs), treatment efficacy is operationalized as the numerical mean of individual treatment successes. In psychotherapy research, efficacy numbers are derived in randomized controlled trials by symptom measurement with validated symptom measures . Validity refers to the test validity of such measures - i.e. to the adequacy of a measure as a means to satisfy its proposed end - which is taken as vital for the validity of the research procedure. However, this paper argues that test validity is principally insufficient as a means for overall epistemic validity of psychotherapy research. Nature of the methodological innovation/critique being proposed: In psychological methodology, validity is strictly bound to instruments, such as the Beck Depression Inventory (BDI ) that has detection of depression symptoms as its end. However, in application the measure becomes a means towards an epistemic end, which may differ from the end of the measure itself. Consequently, the test validity of a symptom measure is only part of the epistemic validity of the operationalization of treatment efficacy per se. In this paper, epistemic validity are discussed both on conceptual and operational level, given that the conceptual level is meaningless in psychotherapy research without its operationalization, yet the operationalization implies multiple validity questions that reach beyond test validity, which implies a paradox in the validity of applied measures in psychotherapy research. This paradox is discussed via an empirical case study from our mixed method psychotherapy study . Two interpretations of its idiosyncratic treatment success are sketched, in which the application of measures implies different conclusions on the test validity given a difference in operationalization of treatment success - which substantiates the need for a concept of validity that goes beyond test validity in psychotherapeutic epistemology. Conclusion and relevance to counselling and psychotherapy research practice: In this paper, the clinical relevance of epistemic validity in psychotherapeutic methodology is highlighted by discussing it within idiosyncratic data. As individual data forms the basis of aggregated data that is used to derive EBTs, which in turn is translated back to individuals in clinical practice, it is vital to understand the epistemic validity of the evidence in Evidence Based Treatments