Hybrid of structural-causal and statistical model for intrusion alert correlation

The evolutions of computer network attacks have urged many organizations to install multiple Network Intrusion Detection Systems (NIDSs) for complete monitoring and detection of intrusions. Such solution produces enormous number of alerts due to repeated and false positive alerts. This contributes to low quality alerts and makes manual Alert Correlation (AC) tedious, labour intensive and error prone. Besides that, alerts are also unformatted, unlabelled and unstructured. Thus, the actual attack strategy cannot be recognized. The existing AC models have few limitations. They only provide single type of correlation and rely on a large number of static predetermined rules to correlate alerts. Consequently, alerts are not being correlated completely and rules need to be manually updated regularly. Therefore, this research proposes a new automated Hybrid-based AC (HAC) model that provides complete correlation in terms of structural, causal and statistical. The purpose is to improve the quality of alerts as well as to recognize the attack strategy through alerts patterns. To accomplish this, it hybridizes Improved Unit Range (IUR), Principal Component Analysis (PCA), Expectation Maximization (EM) algorithm, Levenberg-Marquardt (LM) Backpropagation algorithm and statistical correlation tests to optimally recognize the known and new steps and stages of an attack strategy. New post-clustering algorithms are proposed and become part of the hybridization to filter out the low quality alerts. HAC is successfully experimented using DARPA 2000 benchmark dataset onto signature-based RealSecure Version 6.0 NIDSs. The experimental results validate that HAC optimally correlate the alerts with 98.72% of correlation completeness (Rc) and 3.45 seconds of execution time. This shows that HAC is effective and practical in providing complete correlation even on high dimensionality, large scaled and low quality dataset