Butterfly Identification through a Joint Image-based Recognition and Q&A Learning System

[[abstract]]民眾選擇去大自然的生態旅遊很容易看到蝴蝶，蝴蝶逐漸成為很好的科普教育題材。人們會透過蝴蝶圖鑑去觀察蝴蝶和辨識蝴蝶的種類，然而，這過程對初學來說很費時且不容易尋找到正確的答案，人們困難地辨識蝴蝶物種會使他們降低對蝴蝶的興趣。本論文提出一個辨識蝴蝶的手機應用程式，由兩個子系統組成：一個是影像自動辨識子系統，另一個是Q&A學習子系統。 影像自動辨識子系統可以查詢94種相異的蝴蝶外型，使用方式是對拍攝蝴蝶腹面的生態照之後，對圖片依序點選「蝴蝶後翅的最末端、蝴蝶複眼、前翅頂部末端」三點為影像切割參考點，透過影像正規化、影像去背景、擷取顏色及紋理兩種影像特徵，使用人工神經網路及支持向量機兩種機器學習方式來辨識，實驗結果最好的辨識率是 68.51%。系統使用維基百科的網頁做物種介紹。 當使用者無法從影像自動辨識子系統獲得解答，使用者可以透過上傳照片到蝴蝶Q&A學習子系統去發問，接著專家或是愛好者可以去回答它們。本系統基於群眾外包的方法去做有可信度的物種辨識。此外，發問者可以是一位學習者去嘗試回答別人上傳的問題。學習者可以透過社群得到回饋和獲得分數，當學習者獲得足夠的分數，他們可以升級為專家等級，並且有專家的權限去辨識物種和幫助學習者。[[abstract]]Ecological tourism is the choice of people to go outdoors for nature sightseeing, especially for butterfly species. Butterflies are commonly seen in nature and hence become a good topic for science education. People observe butterflies and used to identify their species through butterfly illustration handbooks. However, this process is time-consuming and hard for beginners to find the correct answer. Difficulties to identify the butterfly species will reduce people’s interests to understand them. In this thesis, we propose a butterfly identification mobile application that combines two complemental sub-systems: an automatic image-based recognition sub-system and a query-and-answer (Q&A) based learning sub-system. We propose an image-based butterfly recognition sub-system to classify 94 kinds of butterflies with dissimilar appearances. The most convenient way to identify butterfly species is to analyze photos of the side views of butterflies. The users select sequentially 3 points on a butterfly image, including end of butterfly hindwing, butterfly compound eye, and end of butterfly forewing. Then our system performs image normalization, image segmentation and feature extraction to obtain the color and texture features. We applied Artificial Neural Network and Support Vector Machine for butterfly classification. The best average precision is 68.51% in the experiments. In this sub-system, introduction of butterflies is re-directed to Wikipedia. When users cannot acquire solutions by the image-based recognition sub-system, users can ask a question by uploading the picture to the butterfly Q&A-based learning sub-system then experts and enthusiasts can answer them. The reliability of the species identification can be done by experts and enthusiasts using a crowdsourcing approach. Moreover, the questioner can become a learner to try to answer questions uploaded by others. The learner can get feedback and earn scores in the community. When learners gain enough scores, they are raised to the Expert-level and have an authority of expert to identify species and help learners.[[note]]碩