Anomaly Detection in Cloud Applications: Internship Report

In very general terms, this internship report consist in analysing data from several experiments on the evolution of a virtual network over time. Using the available data, the objective was to develop a program which could perform online anomaly detection on virtual networks. The available data for this study consists of tables containing information regarding several virtual machines (i.e. the system) which are monitored every 15 seconds. Two sources of data are available: the first source provides data with 150 variables, and the second source with 250. In any case this number of variables is considered as high-dimensional. In addition, a qualitative variable classifies every entry into several possible categories, according to the state of the system, which is either exhibiting normal or anomalous behaviours. In statistical terms, the problem we deal with in this work implies creating a certain decision rule based on the data collected to be able to (accurately) forecast the state of the system (the value of the qualitative variable) knowing the value of the other variables. Therefore this is a classification problem. The main algorithm used for tackling the classification problem is Random forest from the work of Breiman & al