Applications of Image-Based Plant Recognition on Smart Devices and Study on Multiple Descriptors

[[abstract]]本論文提供了一套穿戴式裝置上的應用程式，透過直覺的操作方式提供給使用者植物辨識的服務，在外出時能隨時隨地的使用。此外，也提供另一套在行動裝置上的導覽應用程式，讓在觀光中的民眾可以自發性地去認識、親近植物，以提供認識植物的誘因。 本論文對植物辨識方法做了兩方面的實驗，分別為全域特徵與局部特徵。全域特徵部分，使用 HSV色彩模型來表示花朵的顏色，使用圓形區域色彩直方圖將花朵影像切割成三個圓形的切割區域，以此對花朵影像進行特徵擷取，並用直方圖交集和歐幾里得距離計算相似度。局部特徵部分，特徵區域偵測使用Hessian-Affine，特徵描述子使用了 DAISY、Raw Intensity (RI)、 Geometric Blur (GB)、SIFT、Local Intensity Order Pattern (LIOP) 這五種描述子，此外，本論文再使用將該五種描述子進行混合的多重描述子，期望此描述子能將有上述五種的特性，透過Ranking、Ratio、CAT、Density等四種方法輔以歐幾里得距離進行特徵比對。 本論文的植物影像資料庫為國立暨南大學常見的植物為主，共有 50 種常見的校園植物，每 1 種類的植物皆有 10 張花朵影像，且每 1 張花朵影像皆經過去背，全部共有 500 張影像。為了找出在智慧型裝置上適合的方法，先進行了全域特徵的實驗，特徵比對使用Support Vector Machine (SVM)、Approximate Nearest Neighbor (ANN) 與直方圖交集，而實驗結果顯示，在 Top 10 時 SVM 的辨識率僅有 65 %，ANN的辨識率為76 %，直方圖交集的辨識率為 78%。即使 ANN 與直方圖交集的辨識率僅少 2%，搜尋時間 ANN 只快了 0.82 毫秒，但因為 Top 10未答對的部分使用者需要額外的尋找時間，所以最後在智慧型裝置上的系統決定使用直方圖交集。 在局部域特徵進行實驗的部分，資料集使用 5 對花朵影像，並以人工挑選正確配對。而其中 Pair 1 與 Pair 4 的實驗結果較接近本論文的預期，5 對影像的多重描述子在不同的門檻值下所找到的配對點皆是最多的，只有在門檻值 20 時，Pair 3的結果為 SIFT 比多重描述子高。但因為特徵區域偵測階段時就難以在花朵上找到特徵點，且多聚集於花朵的邊緣或是中心，而花朵的外型是三維空間的架構，所以這些特徵點較難作為花朵的局部特徵使用，以至於既使其中一張有特徵點存在，另一張圖的類似區域也難以找到與之相配對的特徵點存在。[[abstract]]In this thesis, we proposed an application to help users to recognize the plants on wearable devices. With an intuitive operation method, they can use it easily everywhere. Besides, we also proposed a plant guide application on mobile devices which let tourists to understand plants around them. The experiments of plant recognition include two methods: global feature and local feature. For the global feature, HSV color model is used to represent the color of flowers, and the circular color histogram which divide the flower into three circular regions is used to extract features from the flower images. Histogram Intersection and Euclidean Distance are applied to calculate the similarity. For the local feature, region detector applies Hessian-Affine detector, and 5 feature descriptors are used which include: DAISY, Raw Intensity (RI), Geometric Blur (GB), SIFT, and Local Intensity Order Pattern (LIOP). Besides, multiple descriptor fusion is applied for the 5 descriptors. We expect the advantage of distinct properties represented by these descriptors will improve the performance. After single descriptor matching by using Euclidean Distance, four descriptor fusion methods include Ranking, Ratio, CAT, Density are applied. The database of plant images are the common plants in National Chi Nan University. There are 50 species of plants, and each species has 10 flower images. So the database includes 500 images. The background of each flower image is removed. In the experiments of global features, feature matching has applied Support Vector Machine (SVM), Approximate Nearest Neighbor (ANN) and Histogram Intersection. Experimental results show that recognition rate of SVM is 65% in TOP 10 precision, recognition rate of ANN is 76%, recognition rate of Histogram Intersection is 78%. According to the results, the recognition rate of ANN is 2% less than Histogram Intersection, and search time of ANN is 0.82 milliseconds faster than Histogram Intersection. However, users need extra query time when the correct answer is not among the Top 10 results. We decided to use Histogram Intersection on the recognition system of smart devices. For the experiment of local feature, we pick up 5 pairs of flower images, and manually select the correct correspondences from them. Then, the experiment results of Pair1 and Pair4 are like our expectation. Multiple descriptor fusion can find the most correspondences in all 5 pairs of images, but one case is SIFT more than multiple descriptor fusion which is Pair 3 when threshold = 20. According to our observation, it is difficult to find the feature points inside the flower region, so match rate of the dataset is not enough to use, and most of all features cluster on bordering or center of flowers, therefore the feature points don’t describe the flower, because the structure of flower is in three- dimensional space. For example, there is an image pair, one image has some features, but another image doesn't have in the similar area.[[note]]碩