Spark-based Application for Abnormal Log Detection

Ericsson is a world-leader in the rapidly-changing environment of communications technology and thus it is important to provide reliable and high quality networks. Automated test loops are executed frequently, trying to find problems in Ericsson's products but, since test cases alone are not always adequate, machine learning techniques are sometimes used to find abnormal system behaviour. The Awesome Automatic Log Analysis Application (AALAA) tries to find such behaviour by checking the log files produced during the testing, using machine learning techniques. Unfortunately, its performance is not sufficient as it requires a lot of time to process the logs and to train the model. This thesis manages to improve AALAAs performance by implementing a new version that uses Apache Spark, a general purpose cluster computing system, for both the processing of the data and for the training of the machine learning algorithm