Analysis of large correlated data: Application in Statistical Genetics and Spatial Statistics

In the modern age of science, we often confront large, correlated data that necessitates scalable statistical methods to discover the underlying patterns. Traditional statistical tools account for the correlation structure in the data and provide interpretable inference. These methods often do not account for the high volume of the data and hence are not scalable. Modern machine learning algorithms can handle large data, but they do not account for the dependence structure in the data. This thesis attempts to bring the best of both worlds together and develop scalable statistical methods for the analysis of large correlated data with applications in statistical genetics and spatial statistics. On the statistical genetics side, we deal with correlated association statistics. These are available from one-SNP-at-a-time analysis from genome-wide association studies (GWAS). The contribution of the thesis includes the development of a unified analysis of individual- and summary-level data. Here we propose a Biobank level scalable, efficient, joint likelihood-based analysis of individual- and summary-level data from GWAS for heritability estimation. The thesis also develops on partitioning heritability for exploring whether certain regions of the genome, defined by functional annotations, are enriched for association signals. We propose a novel likelihood-based approach for enrichment analysis by explicitly accounting for the sparsity of effect-size distribution. On the spatial statistics side, we develop scalable machine learning models for large spatially correlated data. The first work proposes a highly scalable spatial bootstrap for inference on spatial covariance parameters in Gaussian Process regression which is a notoriously challenging computational problem. The subsequent work focuses on developing a functional estimation method using random forests under dependent error processes. This is motivated by geospatial analysis, where the error process is spatially correlated and modeled using Gaussian Processes. We show that under data dependence, the classical Random forest algorithm does not work well. We propose RF-GLS, a novel and well-principled extension of RF, in the same way, generalized least squares (GLS) fundamentally extend ordinary least squares (OLS) for linear models under dependence. We also establish the asymptotic consistency of RF-GLS and show that it has a better finite sample performance than traditional methods