Pre-processing techniques for anomaly detection in telecommunication networks

Anomalies in telecommunication networks can be signs of errors or malfunctions, which can originate from a wide variety of reasons. Huge amount of data collected from network elements in the form of counters, server logs, audit trail logs etc. can provide significant information about the normal state of the system as well as possible anomalies. Unsupervised methods like ‘Self-Organizing Maps’ (SOM) are often chosen for anomaly detection. They are useful for analyzing and categorizing high volume, high dimensional data. One of the major issues with using SOMs or other unsupervised methods for analyzing anomalies in Telecommunication Management Networks is that they are highly sensitive to pre-treatment of data. The main objective of this thesis is to identify the right kind of pre-processing steps that can be applied to real mobile network traffic data measurements, so that the most interesting anomalies get detected in the anomaly detection stage. Two methods of evaluating the effectiveness of an anomaly detection technique for telecom network measurement data are also proposed