Data-driven framework and experimental validation for security monitoring of networked systems

Cyber attacks have become more prevalent in the last few years, and several attacks have made headlines worldwide. It has become a lucrative business for cybercriminals who are motivated by financial gains. Other motives include political, social and espionage. Organisations are spending a vast amount of money from their IT budget to secure their critical assets from such attacks, but attackers still find ways to compromise these assets. According to a recent data breach report from IBM, the cost of a data breach is estimated to be around $4.24 million, and on average, it takes 287 days to detect and contain such breaches. Cyber attacks are continuing to increase, and no organisation is immune to such attacks, as demonstrated recently by the cyber attack on FireEye, a leading global cybersecurity firm. This thesis aims to develop a data-driven framework for the security monitoring of networked systems. In this framework, models for detecting cyberattack stages, predicting cyber attacks using time series forecasting and the IoC model were developed to detect attacks that the security monitoring tools may have missed. In the cyberattack stage detection, the Cyber Kill Chain was leveraged and then mapped the detection modules to the various stages of the APT lifecycle. In the cyber prediction model, time series based feature forecasting was utilised to predict attacks to help system administrators take preventative measures. The Indicator of Compromise (IoC) model used host-based features to help detect IoCs more accurately. The main framework utilises network, host and IoC features. In these three models, the prediction accuracy of 91.1% and 98.8% was achieved for the APT and IoC models, while the time series forecasting model produced a reasonable low mean absolute error (MAE) and root mean square error (RMSE) score. The author also contributed to another paper on effective feature selection methods using deep feature abstraction in the form of unsupervised auto-encoders to extract more features. Wrapper-based feature selection techniques were then utilised using Support Vector Machine (SVM), Naive Bayes and Decision tree to select the highest-ranking features. Artificial Neural Networks (ANN) classifier was then used to distinguish impersonation from normal traffic. The contribution of the author to this paper was on the feature selection methods. This model achieved an overall accuracy of 99.5%. It is anticipated that these models will allow decision-makers and systems administrators to take proactive approaches to secure their systems and reduce data breaches