IEEE TRANSACTIONS ON NEURAL NETWORKS 1 Multiclass Relevance Vector Machines: Sparsity

Abstract—In this paper we investigate the sparsity and recog-nition capabilities of two approximate Bayesian classification algorithms, the multi-class multi-kernel Relevance Vector Ma-chines (mRVMs) that have been recently proposed. We provide an insight on the behavior of the mRVM models by performing a wide experimentation on a large range of real world datasets. Furthermore, we monitor various model fitting characteristics that identify the predictive nature of the proposed methods and we compare against existing classification techniques. By intro-ducing novel convergence measures, sample selection strategies and model improvements, it is demonstrated that mRVMs can produce state of the art results on multi-class discrimination problems. In addition, this is achieved by utilizing only a very small fraction of the available observation data. Index Terms—Bayesian learning, classification, sparsity, multi-class discrimination, kernel method