Network Anomaly Detection Against Frequent Episodes of Internet Connections

Abstract: New datamining techniques are developed for generating frequent episode rules of traffic events. These episode rules are used to distinguish anomalous sequences of TCP, UDP, or ICMP connections from normal traffic episodes. Fundamental rule pruning techniques are introduced to reduce the search space by 40-70%. Our approach accelerates the entire process of machine learning and profile matching. The new detection scheme was tested over real-life Internet trace data at USC mixed up with 10 days of MIT/LL intrusive attack data set. Our anomaly detection scheme results in a detection rate up to 47 % for DoS, R2L, and port-scanning attacks. These results demonstrate an average of 51 % improvement over the use of association rules alone. We experienced 10 to 20 false alarms over 200 network attacks using scanning windows from 100 sec to 2 hours. Our scheme detects many novel attacks that cannot be detected by Snort, including the Smurf, Apache2, Guesstelnet, etc. This anomaly-based intrusion detection scheme can be used jointly with signature-based IDS to achieve even higher detection efficiency. Index Terms: Network security, intrusion detection, datamining, anomaly detection, connection episodes, false alarms, Internet traffic, and distributed computing 1