The applications of sparsity in classification

The real-world data nowadays is usually in high dimension. For example, one data image can be represented as a thousand to million dimension vector. The disadvantage of processing high dimension data is not only in the term of computational complexity but also in the term of non-reliability due to noisy or corrupted input features. To indentify noisy features, to reconstruct original data from noisy measured model or to perform feature selection, we can reformulate the problem as an energy minimization problem using l0 norm penalty function for the regularization term. It is where the keyword “Sparsity” comes in. Because of the generality of the definition of Sparsity, in this report, we limit our discussion to a particular meaning of sparsity in which we say that a vector is sparse if it has only few non-zero coefficients. From infinitive solution space, basically, we can try to minimize the l0 norm in order to find a sparse solution. However, because sparsity is a general term and it has less meaning without a particular context, in this report, we discuss sparsity in the context of Compressive Sensing and Sparse Support Vector Machine for clarification. The purpose of this report is to demonstrate how sparsity can be used to form a regularization function in minimizing energy function that is applicable to a wide range of practical problem.Bachelor of Engineering (Computer Science