Being idiographic with group data: Seeing is believing without p.

Methodological reform in psychology calls for research to be more idiographic and less dependent on group statistical inference using null-hypothesis significance testing. Recommended alternatives include the use of the new statistics; attention to measurement error, reliable change, Effect Size and clinical/practical significance; more extensive use of graphs and visual analysis; and abandonment of over-reliance on p (e.g., Association for Psychological Science; Cumming; Klein; Task Force on Statistical Inference). This has major implications for applied psychology, given that the application of knowledge is almost always idiographic (i.e., to the single case) while applied research has overwhelmingly been done within the nomothetic, group statistical tradition. This paper describes a synthesis of these alternative approaches to data analysis that presents data on change over time visually for each participant, while presenting group statistics in a way consistent with the new statistics approach. This is done using Modified Brinley Plots, scatter-plots that compare individual scores at time 1 (normally pre-treatment) with scores at various times post-treatment. If the origin and axis scales are the same no or little change is shown by data points clustering on or about the 45o diagonal line. Change associated with treatment (improvement or deterioration) is shown by shifts away from the diagonal. Interpretation is enhanced by the addition of clinical cut-offs, and indicators of means, variances, confidence intervals, measurement error, reliable change, and effect sizes. Both between-group and within-group data may be presented and analysed in this way