Classification of abnormal crowd behavior using image processing and state machines

The study of crowd behavior in public areas or during public events such as subway station, airport and shopping mall had been started two decades ago. In this thesis, an automated video surveillance to detect abnormal activities in a crowd using the concept of state machine is proposed. This method is divided into three stages which are pre-processing, feature extraction and behaviour classification. In preprocessing, frame differencing is used for segmentation while optical flow is performed to estimate the crowd motion. Extracted features consist of global and local features. Global features will consider the features on the whole frame whereas local features only consider the features on each detected object. Based on extracted features, abnormal crowd behaviour can be classified using state machines. The proposed state machine contains four states which will evaluate different features in different states respectively. The frames that are able to reach the final state of the behaviour in its state machine will be classified as the behaviour. The behaviours that can be detected are walking, running, crowd formation, crowd splitting and panic crowd. The method is validated using UMN data set and PETS 2009 data set. The result of the classification has achieved an accuracy of 96.3%