Significance testing: Necessary but insufficient

Sampling fluctuation is an inherent characteristic of data. We cannot pre-tend that it does not exist, nor can we abandon significance testing without a viable replacement. It remains a concern as long as we wish to draw research conclusions within a positivistic paradigm. It is there regardless of whether we conduct a large-group or a single-subject study. It is also there whether we conduct a simple norm-referenced assessment, a com-plex judgment-based assessment, or any other assessment. The widespread misuse of significance testing suggests that sampling fluctuation has been an overbearing guest who has become the inappropriate center of atten-tion at the party. Instead of wishing this guest away, we should put sam-pling fluctuation in its proper place; namely, as but one of the numerous epistemological concerns in any positivistic research. Applied and clinical studies are complex activities. The misplaced emphasis on and the widespread misinterpretation of significance test-ing have made these complex activities more confusing. Unfortunately, Da Prato's (this issue) remedies are misdirected. Abandoning sig-nificance testing (see Note 1) at this time will leave an important gap in any positivistic (or quantitative) research and cast a shadow of doubt on research findings. One of the many questions in any positivistic study is the extent to which sampling fluctuation is the cause of an observed difference. Each single observation of each individual subject in any study is unique. Across subjects, there are natural variations in the behavior or trait of interest. Within each subject, the exact level of trait or behavior fluctuates from time to time and from occasion to occasion. These fluctuations occur naturally without any deliberate interven