[[alternative]]中文語意結構在垃圾信過濾的應用

碩士[[abstract]]為了阻擋垃圾信件, 各界都有所努力, 例如各個國家紛紛成立「反制垃圾郵件法」(「或稱垃圾郵件管制法」), 軟體公司發展防毒以及防惡意程式的軟體等等, 但是即使有再好的防護還是未能完全阻絕。一般使用資料採礦的方法辨別垃圾郵件, 大部分都是從技術方面提升其分類效用, 像是改良分類器或是尋求更好的分類方法,甚少從資料輸入這一部分著手, 本篇論文主要的目的就是透過改善資料輸入的方式, 來使得分類效果提升。 在此考慮了三種類型的輸入變數組合, 除了14個寄件者行為特徵以及20個經由TF-IDF 權重計算所挑選的關鍵字是由前人所提出之外, 我們加入了24個語意成份(也就是各個詞語的詞性) 來表達垃圾郵件寄送者在郵件書寫時的方式。由C4.5、多層感知機以及機率神經網路所驗證的結果來看, 若是加入24個語意成份作為輸入變數, 其效果會比只有14個行為特徵變數加上關鍵字還要好。[[abstract]]In order to prevent spam mails, there are many achievement from the collective efforts of all sectors, although the protections become better and better, the challenges remain. The study focus on how much information is added in the odel, for this reason we hope to explain the output by meliorated version of input elements. We use 14 features of sender’s behavior and 20 keywords which calculated to be the most effectiveness by TF-IDF. Besides that, we proposed 24 new variables of semantic component that simulated the habits of writer and considered the expression between spam e-mail sender and ligitimate e-mail sender. The result shows that simultaneous use of all variables achieve the best results from the point of view of classifiers whatever in C4.5, MLP, or PNN.[[tableofcontents]]Contents Chapter 1 Introduction .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .... .. .. .. .. .. .. .. .. 1 1.1 Background .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 1 1.1 Purpose of Research .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 2 Chapter 2 Literature Review.... .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 3 2.1 Techniques for Filtering Spam .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 3 2.2 Literature Relating to the Classification for Chinese E-mails 6 2.3 The Architecture of E-mail .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 8 2.4 Data Mining Techniques.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 14 2.4.1 C4.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.4.2 Multi-Layer Perceptron (MLP) . . . . . . . . . . . . . . . . . . . 16 2.4.3 Probability Neural Network (PNN) . . . . . . . . . . . . . . . . . 18 2.4.4 TF-IDF Weighting . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.5 The Chinese Word Segmented System .. .. .. .. .. .. .. .. .. .. .. .. .. 22 Chapter 3 Research Method and Procedures .. .. .. .. .. .. .. .... .. .. 23 3.1 The Source of Research Data.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 23 3.2 Research Methods .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 23 3.3 Data Mining Approaches for Classifying the E-mail .. .. .. .. .. 24 3.4 Data Processing.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 28 3.5 Classification Criteria .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 32 3.6 Research Tools .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 33 Chapter 4 Application of Algorithms .. .. .. .. .. .. .. .. .. .... .. .. .. .. .. 34 4.1 Using C4.5 and Variables of Semantic Component to Clas- sify E-mails.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..35 4.2 Using MLP and Variables of Semantic Component to Clas- sify E-mails.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..36 4.3 Using PNN and Variables of Semantic Component to Clas- sify E-mails.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..38 4.4 Summary of the Analysis of the Three Methods .. .. .. .. .. .. .. 39 Chapter 5 Conclusions and Discussion .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 41 Reference .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .... .. .. .. .. .. .. .. .. .. ..42 List of Tables 2.1 Defined the Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Fields of Header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3 Fields of MIME . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.1 League Tables for TF-IDF . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.2 Part of Academia Sinica Balanced Corpus of Modern Chinese . . . . . . . 27 3.3 Summary of all input variables type . . . . . . . . . . . . . . . . . . . . . . 31 3.4 Summary of all output variables type . . . . . . . . . . . . . . . . . . . . . 32 3.5 The result of classification . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.6 The formula for all principles . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.1 Summary of the measures for each situation when using C4.5 . . . . . . . . 35 4.2 Confusion matrix using C4.5 . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.3 Summary of the measures for each situation when using MLP . . . . . . . 37 4.4 Confusion matrix using MLP . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.5 Summary of the measures for each situation when using PNN . . . . . . . 38 4.6 Confusion matrix using PNN . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.7 Summary of the Analysis of the Three Methods . . . . . . . . . . . . . . . 40 List of Figures 2.1 The Process of Sending E-mail . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 ANN with input, hide and output . . . . . . . . . . . . . . . . . . . . . . . 18 2.3 Parzen window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.4 Organization for classification of patterns into categories . . . . . . . . . . 21 3.1 Sample file for mailparsing e-mail . . . . . . . . . . . . . . . . . . . . . . . 28 3.2 The result for using Mailparsing function . . . . . . . . . . . . . . . . . . . 29[[note]]學號: 698650248, 學年度: 9