A short note on resolving singularity problems in covariance matrices

In problems where a distribution is concentrated in a lower-dimensional subspace, the covariance matrix faces a singularity problem. In downstream statistical analyzes this can cause a problem as the inverse of the covariance matrix is often required in the likelihood. There are several methods to overcome this challenge. The most well-known ones are the eigenvalue, singular value, and Cholesky decompositions. In this short note, we develop a new method to deal with the singularity problem while preserving the covariance structure of the original matrix. We compare our alternative with other methods. In a simulation study, we generate various covariance matrices that have different dimensions and dependency structures, and compare the CPU times of each approach