Sparse Bayesian modeling with adaptive kernel learning

Abstract—Sparse kernel methods are very efficient in solving regression and classification problems. The sparsity and perfor-mance of these methods depend on selecting an appropriate kernel function, which is typically achieved using a cross-validation pro-cedure. In this paper, we propose an incremental method for su-pervised learning, which is similar to the relevance vector machine (RVM) but also learns the parameters of the kernels during model training. Specifically, we learn different parameter values for each kernel, resulting in a very flexible model. In order to avoid over-fitting, we use a sparsity enforcing prior that controls the effective number of parameters of the model. We present experimental re-sults on artificial data to demonstrate the advantages of the pro-posed method and we provide a comparison with the typical RVM on several commonly used regression and classification data sets. Index Terms—Classification, kernel learning, regression, rele-vance vector machine (RVM), sparse Bayesian learning. I