Some Problems and Solutions in the Experimental Science of Technology: The Proper Use and Reporting of Statistics in Computational Intelligence, with an Experimental Design from Computational Ethnomusicology

Statistics is the meta-science that lends validity and credibility to The Scientific Method. However, as a complex and advanced Science in itself, Statistics is often misunderstood and misused by scientists, engineers, medical and legal professionals and others. In the area of Computational Intelligence (CI), there have been numerous misuses of statistical techniques leading to the publishing of insupportable results, which, in addition to being a problem in itself, has also contributed to a degree of rift between the Statistics/Statistical Learning community and the Machine Learning/Computational Intelligence community. This talk surveys a number of misuses of statistical inference in CI settings, including well-known and more rarely discussed examples. These are followed by an overview of concepts and techniques that are central to model evaluation. Finally, an experimental design is presented for a statistically valid comparison of multiple hypotheses for a particular real-world problem combining Information Theory, Neural Networks, Statistics, and Computational Ethnomusicology.https://pdxscholar.library.pdx.edu/systems_science_seminar_series/1060/thumbnail.jp