[[alternative]]Labeling model via small sample data -An example of social media posts

[[abstract]]背景動機：近年來網路上文本數據急遽增加，分析文本成為各產業的趨勢，現有的深度學習能靠著文本和標籤進行分類預測，但標籤通常以人工方式進行標註，大量的數據以人工標籤方式並不切實際。另外，深度學習網路皆由多層神經網路層組成，訓練龐大數據相當耗時，然而在沒有這麼多標籤的情況下要建立實時且準確的標註模型非常困難。目的：為解決上述問題，本研究以小樣本建立標註模型，分別以兩個面向解決此問題，分別是數據建模前的提前篩選以及深度學習之小樣本模型，設立兩項目標：1.使用抽樣策略，減少模型訓練時所需樣本數量2.使用小樣本建立標註模型時，選擇最佳的深度學習法作為建模方法。方法：本研究收集了2019-2020社群網站PTT Prozac版上文章作為數據，並以該文章分類作為模型訓練時的標籤，刪除了兩個標籤以外的文章，另外，文章最大字數只擷取至512個字元，字數超過的文章只保留前512個字元。數據經篩選後留下了1680筆文章，將其拆分為訓練數據840筆和測試數據840筆。處理文章以文本表徵呈現使用了BERT作為萃取工具，將一篇文章以768維度表示，在訓練模型前，處理過後的樣本透過六種選擇策略(熵、吉尼係數、分群、熵+吉尼係數、熵+吉尼係數+分群、隨機)分別以不同數量進行樣本的篩選，最後在遷移學習、主動學習、多任務學習以及元學習上建立標註模型，所有模型的優化函數為Adam，損失函數為交叉熵，學習率3e-6，批次大小8，以上數值在每個模型中迭代10次。並以準確度、F1分數、召回率以及精確度做為評估標準。因此每種標註模型將會產生120種結果，協助判別各種組合在小樣本上的優勢與劣勢。結果：以方法進行實驗產生的120種數值表示，事先進行數據選擇的準確度，能夠比隨機抽樣高上3個百分點，並且使用熵作為抽樣方法最為穩定；另外，四種標註模型中，以元學習的準確度最高，在完整數據建模的準確度84.64%之下，元學習僅使用20%的樣本建模，準確度可達到78.45%。討論與結論：研究發現使用熵進行數據選擇並以元學習訓練模型，只需20%的數據能夠達到完整數據建模準確度的9成，透過小樣本建模達到高水準的標註能力，除了能解決模型的運算時間之外，人工介入標籤的行為也大幅減少，並且以元學習的特性來說，其模型的參數還可以應用在不同領域中，大幅減少在個別任務中重新訓練參數的繁雜步驟，而與元學習同樣使用額外數據集訓練的多任務學習準確度較低，與文獻顯示一致，多任務學習較易受到不同類型的任務所影響，因此以元學習建立小樣本模型最佳。未來以小樣本建模的問題應還會持續存在，若自然語言處理尚未出現新技術，建議可以多嘗試多種特徵選擇策略，在建立模型前就使用相對具有建模意義的數據，增加模型準確度。[[abstract]]Background: In recent years, the text data on the Internet has increased rapidly, and analyzing text has become a trend in various industries. Deep learning can rely on text and labels for classification and prediction, but labels are usually labeled manually, and manual labeling of large amounts of data is not practical. In addition, deep learning networks consist of multiple neural network layers, and training on massive data is time-consuming. However, without so many labels, it is difficult to build accurate annotation models in real-time.Methods: This study collected articles from the 2019-2020 community site PTT Prozac version as data. When the model is trained, the article classification is used as the label, and the articles other than the two labels are deleted. Also, only the maximum word count in the article is extracted. Keep the first 512 characters of the article at most, if it exceeds, delete it. After the data was filtered, 1,680 articles were left, which were split into 840 for training data and 840 for testing data. The processed articles are presented as text representations. BERT is used as an extraction tool to represent an article in 768 dimensions. Before training the model, the processed samples are processed through six selection strategies (entropy, Gini coefficient, clustering, entropy + Gini coefficient, entropy + Gini coefficient + clustering, random) to screen samples with different numbers, and finally establish a labeling model on transfer learning, active learning, multi-task learning and meta-learning. The optimization function of all models is Adam, and the loss function is Cross-entropy, learning rate 3e-6, batch size 8, the above values are iterated 10 times in each model. Accuracy, F1 score, recall and precision are used as evaluation criteria. Therefore, each labeling model will generate 120 kinds of results to help identify the advantages and disadvantages of various combinations on a small sample.Results: According to the 120 kinds of numerical values generated by the experiment of this method, the accuracy of selecting data in advance can be 3 percentage points higher than that of random sampling, and entropy as the sampling method is the most stable; meta-learning has the highest accuracy. Under the accuracy rate of 84.64% for complete data modeling, meta-learning only uses 20% of the sample modeling, and the accuracy rate can reach 78.45%.Discussion and conclusions: The study found that using entropy for data selection and using meta-learning to train models can achieve 90% of the accuracy of complete data modeling with only 20% of the data. In addition to solving the calculation time of the model, the behavior of manual intervention of labels is also greatly reduced, and in terms of the characteristics of meta-learning, the parameters of the model can also be applied in different fields, greatly reducing the complicated steps of retraining parameters in individual tasks, and the multi-task learning that uses additional datasets to train like meta-learning has lower accuracy. Consistent with the literature, multi-task learning is more susceptible to different types of tasks, so it is best to use meta-learning to build a small sample model. In the future, the problem of modeling with small samples will continue to exist. If there is no new technology in natural language processing, it is recommended to try a variety of feature selection strategies. Before building a model, use relatively meaningful data to increase the accuracy of the model