A Fresh Look at Combining Logs and Network Data to Detect Anomalous Activity

As data rates have increased, network administrators have increasingly turned to Software Defined Networking (SDN) to increase efficiency, as well as to react quicker to changing network states. However, as SDN flows become the norm to manage network traffic, Network Intrusion Detection Systems (NIDS) still rely on processing packet data directly using techniques such as Deep Packet Inspection (DPI). SDN flows provide only a high level representation of the packets traversing the network, reducing the amount of data available to NIDS. In particular Deep Learning based NIDS may be affected. Deep Learning has been proposed as a solution to 0-day attacks, but these models typically require large volumes of training data with many data points. This paper proposes a solution to this dilemma, by providing more data points for an IDS to monitor through the abstraction of log data generated by the flows. Past papers have shown that the quality of training data can have a marked effect on performance of Deep Learning models. This paper builds on these works by showing that high quality data points can be added in a computationally inexpensive manner, and through adding these data points, accuracy on a real world dataset can be increased by upwards of 1