Improved quality metrics for association and reproducibility in chromatin accessibility data using mutual information

Abstract Background Correlation metrics are widely utilized in genomics analysis and often implemented with little regard to assumptions of normality, homoscedasticity, and independence of values. This is especially true when comparing values between replicated sequencing experiments that probe chromatin accessibility, such as assays for transposase-accessible chromatin via sequencing (ATAC-seq). Such data can possess several regions across the human genome with little to no sequencing depth and are thus non-normal with a large portion of zero values. Despite distributed use in the epigenomics field, few studies have evaluated and benchmarked how correlation and association statistics behave across ATAC-seq experiments with known differences or the effects of removing specific outliers from the data. Here, we developed a computational simulation of ATAC-seq data to elucidate the behavior of correlation statistics and to compare their accuracy under set conditions of reproducibility. Results Using these simulations, we monitored the behavior of several correlation statistics, including the Pearson’s R and Spearman’s $$\rho$$ ρ coefficients as well as Kendall’s $$\tau$$ τ and Top–Down correlation. We also test the behavior of association measures, including the coefficient of determination R $$^2$$ 2 , Kendall’s W, and normalized mutual information. Our experiments reveal an insensitivity of most statistics, including Spearman’s $$\rho$$ ρ , Kendall’s $$\tau$$ τ , and Kendall’s W, to increasing differences between simulated ATAC-seq replicates. The removal of co-zeros (regions lacking mapped sequenced reads) between simulated experiments greatly improves the estimates of correlation and association. After removing co-zeros, the R $$^2$$ 2 coefficient and normalized mutual information display the best performance, having a closer one-to-one relationship with the known portion of shared, enhanced loci between simulated replicates. When comparing values between experimental ATAC-seq data using a random forest model, mutual information best predicts ATAC-seq replicate relationships. Conclusions Collectively, this study demonstrates how measures of correlation and association can behave in epigenomics experiments. We provide improved strategies for quantifying relationships in these increasingly prevalent and important chromatin accessibility assays