Expandable Factor Analysis

Bayesian sparse factor models have proven useful for characterizing dependencies in high-dimensional data. However, lack of computational scalability to high-dimensions (P) with unknown numbers of factors (K) remains a vexing issue. We propose a framework for expandable factor analy-sis (xFA), where expandable refers to the ability of scaling to high-dimensional settings by adaptively adding additional factors as needed. Key to this behavior is the use of a novel multiscale generalized double Pareto (mGDP) prior for the loadings matrix. The mGDP prior is carefully structured to induce sparsity in the loadings, allow an unknown number of factors, and produce an objective function for maximum a posteriori estimation that factorizes to yield P separate weighted `1-regularized regressions. Model averaging is used to remove sensitivity due to the form of mGDP prior and number of factors. We provide theoretical support, develop efficient computational algorithms, and evaluate xFA’s perfor-mance using simulated data and genomic applications. Computational efficiency is further improved via one-step estimation. Key words: Bayesian model averaging; covariance matrix; EM-type algorithm; factor analysis; general-ized double Pareto; high-dimensional; Laplace approximation; non-concave variable selection; sparsity.