Online Anomaly Detection in Crowd Scenes via Structure Analysis

Abstract—Abnormal behavior detection in crowd scenes is con-tinuously a challenge in the field of computer vision. For tackling this problem, this paper starts from a novel structure modeling of crowd behavior. We first propose an informative structural context descriptor (SCD) for describing the crowd individual, which originally introduces the potential energy function of par-ticle’s interforce in solid-state physics to intuitively conduct vision contextual cueing. For computing the crowd SCD variation effec-tively, we then design a robust multi-object tracker to associate the targets in different frames, which employs the incremental analytical ability of the 3-D discrete cosine transform (DCT). By online spatial-temporal analyzing the SCD variation of the crowd, the abnormality is finally localized. Our contribution mainly lies on three aspects: 1) the new exploration of abnormal detection from structure modeling where the motion difference between individuals is computed by a novel selective histogram of optical flow that makes the proposed method can deal with more kinds of anomalies; 2) the SCD description that can effectively rep-resent the relationship among the individuals; and 3) the 3-D DCT multi-object tracker that can robustly associate the lim-ited number of (instead of all) targets which makes the tracking analysis in high density crowd situation feasible. Experimental results on several publicly available crowd video datasets verify the effectiveness of the proposed method. Index Terms—Anomaly detection, computer vision, machine learning, object tracking, structure analysis, video analysis. I