Streetscape Recognition System

研究主要透過影像處理的方式，回報使用者目前的位置資訊，主要的影像識別技術由SIFT (Scale-invariant feature transform)作為核心，擷取影像的特徵點後再進行特徵匹配。雖然SIFT對於影像的亮度變化有良好的穩定性，但對於亮度反差過大的街景影像還是無法處理的很好，若直接以直條圖等化 (Histogram Equalization)對影像作調整，則影像中對比度較差的區域會產生過多的雜訊，造成比對時的負擔。因此本研究提出了「條件式」的直條圖等化 (Histogram Equalization) 來降低亮度變化對於影像比對的影響，以提升街景比對之成功率。此外，針對原著所提出的特徵點匹配演算法BBF (Best Bin First)也做了相當程度的修改，藉由不同的參數對K-D Tree進行搜尋，可以得到不同程度的特徵點匹配點數與執行時間。 藉由SIFT進行特徵點匹配後，可得到兩影像特徵點匹配數目，在資料庫中得到最多特徵點匹配數者並不一定是最鄰近來源影像的街景，因為有可能來源影像與資料庫中的街景皆不相鄰，但還是可以得到近似於0之特徵點匹配數目，因此在最後街景驗證過程中，將每張街景影像加入了「鄰近景點的資訊」，透過特「徵點匹配數」與「鄰近景點資訊」作為驗證的依據，除了可改善上述的情況發生，提升街景回報的準確率之外，還能將使用者所在位置描述得更明確，如使用者所在位置靠近某景點，或是處於某兩景點之間等等。 利用本研究之方法所實作的系統，在最後分別以不同的觀點及拍攝環境來呈現多個實驗結果。在綜合實驗結果中，將測試資料日、夜間街景1000張與資料庫中的街景影像做比對，在日間的正確率98.2%，夜間的正確率為95.8%。The feature of this research is to report the current position of the user by image processing. The main core of this image recognition technology is SIFT (Scale-invariant feature transform) which runs feature matching after extracting the features from the images. Although SIFT has stable performance on illumination changes of images, it still shows discrepancies when contrast gets too high. It will produce more noises if using the Histogram Equalization method directly. In order to increase accuracy, this research runs conditionally Histogram Equalization which can decrease the impact of high contrast gap. In addition, my algorithm is modified from the original which has certain amount of the BBF (Best Bin First). My algorithm also obtains features outcomes and process time by searching K-D Tree with different arguments. This main idea is to obtain the numbers of features of two images after running SIFT; however, the highest numbers of features coming up from database does not mean it is the right results. Sometimes the matching numbers of features shows a number which is close to zero, the source image does not match the database; therefore, the last verification process assigns “nearby scene information” to every street image by verifying with both “numbers of features” and “nearby scene information” so that it not only improves accuracy but also reports a more correct user location; for instance, it will show if the user is very close to a specific scene or located in the middle of certain two scenes. This system we propose will generate various outputs base on different viewpoints and environments. Under 1000 different testing data for both daytime and night time to obtain matching results from current database, our research runs for 98.2% and 95.8% accuracy