One-class conditional random fields for sequential anomaly detection

Sequential anomaly detection is a challenging problem due to the one-class nature of the data (i.e., data is collected from only one class) and the tem-poral dependence in sequential data. We present One-Class Conditional Random Fields (OCCRF) for sequential anomaly detection that learn from a one-class dataset and capture the temporal depen-dence structure, in an unsupervised fashion. We propose a hinge loss in a regularized risk mini-mization framework that maximizes the margin be-tween each sequence being classified as “normal” and “abnormal. ” This allows our model to accept most (but not all) of the training data as normal, yet keeps the solution space tight. Experimental results on a number of real-world datasets show our model outperforming several baselines. We also report an exploratory study on detecting abnormal organiza-tional behavior in enterprise social networks.