Latent Dirichlet Allocation、classification、feature scaling、positive and negative data

[[abstract]]分類問題中，假如我們只在意將資料分到目標類別，就稱為one-class分類。在這種環境中，目標集合以外的資料就稱為outliers，而本篇論文將去探討在one-class中，如果outliers數量遠超過目標集合且分佈極度離散的狀況下，該如何去處理並且正確地將資料分到目標類別？ 以KDD Cup 2008為例，1712位病患中只有118位患有乳癌，而每位病患都以許多筆candidates來描述他的情況，總共102,294個candidates，其中只有623個是positive，也就是我們所謂的目標集合，如何在正負樣本落差極大的情形下，正確地將正常病患與罹癌患者區分開來呢？我們將以KDD Cup 2008年的挑戰嘗試解決這樣的問題，藉以強化我們的LDA model。 從實驗結果可以發現經過我們調整以及去除雜訊的步驟之後，分類的準確率比沒有經過調整和去除雜訊的資料準確率進步許多，由此可見這兩種程序步驟的確可以強化LDA model，面臨這類問題時，能有效地增加分類的準確率。[[abstract]]Among various classification problems, the one-class problem focuses on how to correctly classify data into target class. The others outside target class are called outliers. In this paper, we are going to discuss the classification for extremely biased data set the amount of outliers is far larger than target set of data, and the outliers are discretely distributed. We use KDD Cup 2008 challenge as our target experiment. There are 1,712 patients, only 118 of them are malignant of breast cancer. Each patent’s data is represented as multiple candidates records. In summary, there are 102,294 candidates and only 623 positive (positive) samples. These positive samples are our target set for classification. Under the above data distribution, how to separate normal data with malignant data is a pretty difficult job. In this paper, we propose a classification process flow to solve this problem