Support Vector Machines: Classification with Coding and Regression for Gene Selection

本論文主要分為兩部分。在第一部份中，著重於利用編碼(coding)找出一個低維線性分類子空間(low-dimensional linear discriminant feature subspace)的方法，並探討不同編碼之間的等價性質(equivalence)。透過編碼的方法可以將類別(class label)轉換成多維反應量(multiresponse)，將此多維反應量與核化資料(kernelized data)進行迴歸分析，再進一步利用迴歸係數得到低維線性分類子空間。此子空間可結合任意的線性分類法，使計算較為簡潔快速。在這一部份中也證明，任意編碼產生的多維反應量都會生成同樣的低維線性分類子空間，因此任意的線性分類法都會得到相同的分類結果。實際資料分類的結果顯示，本文提出的分類方法與LIBSVM比較，具有相近的正確率，但是需要較少的分類時間。第二部分中，本文提出了一個利用支撐向量迴歸(support vector regression)進行基因選取(gene selection)的方法。目前根據微陣列資料(microarray data)作基因選取的方法都將每一片生物晶片視為相同。然而，生物晶片也許來自於不同疾病狀態的病人身上，因此與疾病的相關也不全然相同。所以應當給予生物晶片不同的權重來表示這些生物晶片與疾病之間的相關性。而這些權重可以由支撐向量迴歸估計得來。將這些加權過後的表現(weighted expressions)相加後得到的數值，可以用來決定哪些基因是有顯著意義的基因(significant genes)。我們使用白血病(leukemia)與結腸癌(colon cancer)的資料作分析，並比較其他基因選取的方法所得之正確率。結果顯示，本文提出的基因選取方法可以找出有顯著意義的基因。This thesis contains two major themes. One is the multiclass support vector machines and the other is the support vector regression for gene selection. In the first part, we propose a regression approach for multiclass support vector classification. We introduce some existing coding schemes into the support vector classification by coding the class labels into multivariate responses. Regression of these multivariate responses on kernelized input data is used to extract a low-dimensional featureubspace for discriminant purpose. We unify these coding schemes by showing that they are equivalent in the sense of leading to the same low-dimensional discriminant feature subspace. Classification is then carried out in this low-dimensional subspace using a linear discriminant algorithm, which can be any reasonable choice. The regression approach for extracting low-dimensionaliscriminant subspace combined with user-specified linearlgorithm can team up into a simple but yet powerful toolkit for multiclass support vector classification. Issues of encoding, decoding and the notions of equivalence of codes are discussed. Experimental results, including prediction ability and CPU time, show that our approach is a competent alternative for the multiclass support vector machine problem.n the second part, we propose a support vector regressionpproach for gene selection and use the selected genes for disease classification. Current gene selection methods based on microarray data have treated each individual subject with equal weight to the disease of interest. However, tissues collected from different patients can be from different disease stages and may have different strength of association with the disease. To reflecthis circumstance, our proposed method will take into account the subject variation by assigning different weights to subjects. The weights are calculated via support vector regression. Then significant genes are selected based on the cumulative sum of weighted expressions. The proposed gene selection procedure isllustrated and evaluated using the acute leukemia and colon cancer data. The results and performance are compared with four other approaches in terms of classification accuracies.1 Introduction 1 Preliminaries: Support Vector Machines 3.1 Linearly separable case . . . . . . . . . . . . . . . . . . . 3.2 Linearly non-separable case . . . . . . . . . . . . . . . . 6.3 Nonlinear extension by kernel trick . . . . . . . . . . . . 8.4 Smooth support vector machine . . . . . . . . . . . . . . 9.5 Extension to multiclass classification problem . . . . . . 11.5.1 One-against-rest and one-against-one . . . . . . . 12.5.2 Single machine approach . . . . . . . . . . . . . . 13.6 Support vector regression . . . . . . . . . . . . . . . . . . 18.7 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.7.1 LIBSVM. . . . . . . . . . . . . . . . . . . . . . . 21.7.2 SSVM toolbox . . . . . . . . . . . . . . . . . . . . 21 Classification by Coding and Multiresponse Regression 23.1 Regression framework: linear and kernel generalization . 24.2 Regularized least-squares support vector regression . . . 27.3 Decoding and classification rules . . . . . . . . . . . . . . 28.4 Encoding and equivalence class of codes . . . . . . . . . 30.4.1 Coding and scoring schemes . . . . . . . . . . . . 30.4.2 Equivalence class of codes . . . . . . . . . . . . . 33 Application to Benchmark Data Sets 37.1 Benchmark data sets . . . . . . . . . . . . . . . . . . . . 37.2 Comparisons of coding schemes . . . . . . . . . . . . . . 42.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 45 Gene Selection with Support Vector Regression 51.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Current methods . . . . . . . . . . . . . . . . . . . . . . 53.2.1 SVM-based recursive feature elimination . . . . . 54.2.2 Incremental forward feature selection . . . . . . . 55.2.3 Bayesian variable selection . . . . . . . . . . . . . 55.2.4 Bayesian model average . . . . . . . . . . . . . . 56.3 Proposed gene selection method . . . . . . . . . . . . . . 57.4 Empirical data analysis . . . . . . . . . . . . . . . . . . . 61.5 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . 67 Discussions and Future Directions 69eferences 7