Statistical Theory and Methods for Comparing Distributions

With the recent advancement of data collection techniques, there has been an explosive growth in the sizeand complex of data sets in many application domains. The rise of such unprecedented data has posed newchallenges as well as new opportunities to researchers in statistics and data science. Traditional methods,tailored to static and low-dimensional data, perform poorly or are no longer applicable for modern high dimensionaldata with complex structures. Moreover, classical asymptotic theory easily breaks down under non-traditional settings where numerous parameters can interact in dynamic ways. Motivated by these new challenges, this dissertation aims to develop novel methods and technical tools suitable for modern high dimensional data with particular emphasis on three types of testing problems: (i) one-sample testing, (ii)two-sample testing and (iii) independence testing.One of the major contributions of this thesis is to introduce a exible two-sample testing framework that can leverage any existing classi?fication or regression method. By taking advantage of state-of-the-art algorithms in machine learning, the proposed method can efficiently handle different types of variables and various structures in high-dimensional data with competitive power under a variety of practical scenarios. To justify our approach, we provide rigorous theoretical and empirical analysis of their performance. With a speci?fic focus on Fisher's linear discriminant analysis, we prove more sophisticated results including minimaxoptimality under common regularity conditions. In addition to supervised learning approaches, we also contribute to the literature by proposing goodness-of-?t tests for high-dimensional multinomials as well as multivariate generalizations of classical rank-based tests.Another theme of this dissertation is concerned with permutation tests. Although the permutationapproach is standard in practical implementations of two-sample and independence testing, its theoreticalproperties, especially power, have not been explored beyond simple cases. A major challenge of analyzing thepermutation test is that it depends on a random critical value which is a function of observations. We studyhow to overcome this challenge and demonstrate that the permutation test has competitive power propertiesfor many interesting problems under non-traditional settings. In particular we use the minimax perspectiveto evaluate the performance of a test and show that the permutation test is optimal for the problems whereminimax lower bounds are available.