Evaluation of SVD and NMF methods for Latent Semantic Analysis

Abstract. Different mathematical techniques are being developed to reduce the dimensionality of data within large datasets, for robust retrieval of required information. Latent Semantic Analysis (LSA), a modified low rank approximation form of Vector Space Model, can be used for detecting underlying semantic relationships within text corpora. LSA performs a low-rank approximation on term-document matrix, which is generated by transforming textual data into a vector representation, thereby bringing out the semantic connectedness present among the documents of the corpus. Singular Value Decomposition (SVD) is the traditional approximation method used for LSA, wherein lower dimensional components from the decomposition are truncated. On truncation, the linguistic noise present in the vector representation is removed, and the semantic connectedness is made visible. One of the pitfalls of using SVD is that the truncated matrix will have negative components, which is not natural for interpreting the textual representation. Nonnegative Matrix Factorization (NMF) addresses this issue by generating nonnegative parts-based representation as the low rank approximation for performing LSA. The paper provides an in-depth overview of how both methods are being used for the purpose of Information Retrieval. Performance evaluation of the methods has been performed using standard test datasets