Replication data for: Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance

For over a half-century, various fields in the behavioral and social sciences have debated the appropriateness of null hypothesis significance testing (NHST) in the presentation of research results. A long list of criticisms has fueled the so-called significance testing controversy. The conventional NHST framework encourages researchers to devote excessive attention to statistical significance while underemphasizing practical (scientific, substantive, social, political, etc.) significance. I introduce a simple, intuitive approach that grounds testing in subject-area expertise, balancing the dual concerns of detectability and importance. The proposed practical and statistical significance test allows the social scientist to focus upon real-world significance, taking into account both sampling error and an assessment of what parameter values should be deemed interesting, given theory. The matter of what constitutes practical significance is left in the hands of the researchers themselves, to be debated as a natural component of inference and interpretation