Algorithm and Architecture Design of Vehicle Tracking for Intelligent Vehicle Cruise Control System

本篇論文針對未來以視覺為主的車用智慧巡航系統，提出一個以知識為基本（knowledge-based）的智慧型車輛追踨系統來滿足安全性與節能的需求。在未來智慧車發展的趨勢下，安全與節能是其兩大重要發展目標，而車用智慧巡航系統即是一個同時滿足安全與節能的重要應用，而趨勢同時也顯示，隨著影像分析技術的發展，電腦視覺將會在智慧型巡航功能中扮演重要的角色。本篇論文以視覺為主的車用智慧巡航系統為研究目標，就功能、成本、系統規格，三方面進行車輛辨識與追蹤系統討論與開發。 本論文先探討影像規格對車用巡航系統的影響。透過行車能源效率的模擬，發現在自動巡航的控制下，對周圍車輛相對速度偵測的準確度會直接影響到能源的使用效率，而相對速度偵測的準確度則受到影像規格本身的影響，在同時考量車用安全性與節能效率的條件下，我們預期系統最高將支援到超高畫質（4096x2160）的解析度，並在該解析度下達到每秒十張畫面以上的運算速度。 而本論文接續對一車輛辨識與追踨系統進行探討。在車輛辨識與追蹤系統中，車輛辨識模組負責辨識出畫面中車輛的位置並輸出給車輛追蹤模組進行追蹤與距離偵測。雖然以學習為基本（learning-based）的辨識演算法在表現上有較高的辨識率與可靠度，其在物體的定位、物體的大小判斷以及錯誤偵測率上仍有其演算法上的極限，而在面對這些誤差的條件下，我們對幾種常見的物體追蹤演算法進行討論，發現現有追蹤演算法在功能性上並無法解決錯誤起始定位的問題，故無法真正應用於視覺巡航的系統中， 因此，我們提出一個以知識為基本（knowledge-based）的智慧型車輛追蹤演算法，以車子本身的存在的特點進行演算法開發，「定位自動校準」以及「錯誤偵測消除」為其兩大功能，實驗結果顯示，所提出以知識為基本的演算法擁有較佳的距離偵測能力並且能有效消除錯誤偵測的發生，且由於其能容忍起始定位的偏移誤差，我們使用辨識與追蹤的平行處理來取代連續式的運算，將可大量省下系統所需記憶體成本。 在高畫質畫面下達到系統要求的處理能力亦是系統之重要要素，最後一部分，我們對所提出的追蹤演算法進行執行時間的分析與最佳化，並對其提出硬體加速架構，在設計的同時，不同的最佳化技巧亦被應用在硬體的架構上。而在使用硬體加速後的系統在1280x960 的畫面解析度下最快可達到每秒81.4 張影像的處理速度，而支援到最高4096x2160 的畫面解析度亦可達到每秒11 張影像的處理速度，符合巡航控制所需規格條件。我們硬體最後實作於UMC 90nm Logic Low-K SP-RVT Process 製程，面積為2.2 x 2.2 mm2，記憶體大小為12.8Kbits，操作頻率為100MHz，所需功率最低為23.45mW、最高為648.75mW，最多可同時支援五個目標的追蹤。In this thesis, future vision-based intelligent cruise control system is targeted. A knowledge-based intelligent vehicle tracking system is proposed to satisfy the demands of safety and energy-saving purposes. In the trend of future intelligent vehicle, safety and energy efficiency are two critical developing goals. Intelligent cruise control, which satisfies both safety and energy efficiency purposes, is an important vehicle application for that. The developing trend also shows, with the enhancement of video analysis technologies, computer vision will play an important role in the future cruise system. Therefore, this thesis explores and researches on a vision-based vehicle cruise. A vehicle recognition and tracking system is discussed and developed by three aspects of performance, cost, and system specification. First the impact of video specification on vehicle cruise control is discussed. By the energy consumption simulation for driving cycles, we can noticed that in automatic cruising condition, the detection accuracy of relative velocities of surrounding vehicles directly affects the energy efficiency. However, the velocity detection accuracy is influenced by the video specification. Considering the vehicle safety and energy efficiency issues, we expect that the proposed system can support up to super high resolution (4096x2160), and 10-frame-per-second computing throughput can be reached under that resolution. The system of vehicle recognition and tracking is explored as follows in this thesis. In the system, the module of vehicle recognition is responsible to recognize vehicle positions in image and output them to the vehicle tracking modulefor tracking and range detection. Though learning-based recognition algorithms perform high recognition rate and reliability, there are limitations on the abilities of object positioning, object size determination and false alarm rate minimization. Facing to these error conditions, several common object tracking algorithms are discussed. It can be found that current related algorithms are incapable of solving the problem of error position initialization and cannot be practically applied in vision-based cruise system. Therefore, a knowledge-based intelligent vehicle tracking algorithm is proposed. The algorithm is developed with the existed knowledge of vehicle characteristics, and it possesses two main functionalities, position auto-adjustment and false alarm reduction. As experimental results show, the proposed knowledge-based algorithm performs better ability of range detection, and it also effectively reduces the system false alarm rate. Moreover, due to the resistance against the departure error of initial position, a recognitionand- tracking parallel processing scheme can be applied instead of the sequential processing, which reduces a large amount of system memory cost. One of the essential factors is the required system throughput under the targeted high image resolution. In the last part, the execution timing performance of proposed tracking algorithm is analyzed. A hardware-oriented algorithm optimization methodology and its corresponding hardware architecture are proposed for acceleration. During hardware design, different optimization techniques are applies to the architecture. The system after hardware acceleration reaches the processing speed of 81.4 frames per second under 1280x960 image resolution, and it can support up to 4096x2160 image resolution with 11 frames per second processing speed, which fits the specification of cruise control system. The hardware is finally implemented with UMC 90nm Logic Low-K SP-RVT Process technology. The total chip size is 2.2x2.2mm2 with 12.8Kbits on-chip memory. Operating frequency is 100MHz and the minimum and maximum powers are 23.45mW and 648.75mW, respectively. Maximum five targets can be tracked simultaneously