Video based vehicle classification methods

© 2014 Mandar SovaniVehicle classification is a significant problem in transportation management, planning, surveillance and security systems. Traditionally the Pneumatic tube and induction loop sensor based systems are used for vehicle classification. These sensors are not only expensive and invasive but also difficult to maintain. This has led to the adoption of several non-invasive technologies like laser, video and TIRTL which are easy to install and maintain. Both laser and TIRTL provide vehicle classification information based only on size/length and they often fall short of classifying vehicles completely. On the other hand video can be used to extract features like color and license plate along with the size and shape of the vehicle to determine its class. This thesis proposes robust methods to accurately solve the vehicle classification problem by extracting the aforementioned features from video under highly uncontrolled real life environments. The thesis begins by introducing the problem definition and scope along with the review of current methods for vehicle classification in Chapter 1. Most popular video based solutions use background estimation algorithms on color/IR illuminated cameras to detect the vehicles and then classify according to size and shape extracted from the foreground objects. Accuracy of these methods is heavily dependent on the accuracy of foreground extraction methods which in turn is dependent on the quality of the input video stream. The traditional color and IR cameras used in such applications are not designed to deal with strong shadows, sudden illumination changes, headlights and other such effects leading to errors in foreground extraction and consequently errors in vehicle classification. To overcome the limitation of color and IR cameras, this thesis proposes the use of thermal cameras to detect and classify vehicles. This is covered in chapter 2 of the thesis along with the track-before-detect technique to reduce false positives and track multiple vehicles. While the proposed size based classification using thermal cameras led to several improvements, the experimental results demonstrate that it is not a completely reliable method for vehicle classification. By noting the fact that additional information about vehicle class is encoded in the form of license plate color and in some cases in the license plate number pattern, we explore methods to accurately infer the vehicle class by using these features in chapter 3. For example in Singapore, license plates starting with P indicate buses and in India license plates with yellow color indicate commercial vehicles. This chapter proposes a feedback based method to improve the license plate image binarization for character segmentation which helps in extracting the number plate pattern accurately. It also explores a distance to color based method on the license plate for a limited color classes without any constancy transformations. However, there are several unresolved ambiguities associated with these approaches. For instance, although most vehicles with specified colors belong to a particular class, not all belong to the same class. Same is the case with prefixes and number plate patterns. To overcome these shortcomings fusion of size and feature based methods are proposed in Chapter 4. We adopt multi-feature Bayesian Fusion approach to fuse the information from vehicle size and license plate information to deliver least ambiguous and most accurate vehicle classification results. Significant effort of the work reported in the thesis was put into implementing the proposed methods in live field conditions, test, verify and validate their performance. Detailed descriptions of on-field experiments carried at various live sites, including Rajiv Gandhi International Airport, Hyderabad India, West Coast Way, Singapore and Abu Dhabi Airport, UAE are summarized in Chapter 5. Based on these experiments and results obtained we conclude that the fusion of multiple features through recursive Bayesian approach provides the most robust and accurate vehicle classification solution. It concludes with key observations and future research directions