System of intelligent methods for predicting failures in broadband networks

Cilj doktorskog rada je istraživanje mogućnosti kratkoročnog i dugoročnog predviđanja očekivanog broja smetnji u širokopojasnim telekomunikacijskim mrežama upotrebom različitih modela predviđanja. Istraživanje je provedeno na konkretnim podacima žive telekomunikacijske mreže u vremenskom periodu 2009.-2012.; broj smetnji uključenih u analizi prelazi 2,5 milijuna. Fokus istraživanja je stavljen na dinamičke neuronske mreže specijalizirane za predviđanja u nelinearnim sustavima, dok se u svrhu komparacije uključuju i tradicionalne autoregresivne metode predviđanja. Podešavanjem varijabilnih parametara pojedinih modela pokušava se postići maksimalna točnost predviđanja. Najbolje rezultate predviđanja daje metoda temeljena na rekurzivnim neuralnim mrežama koja pored vremenskih serija povijesnih podataka koristi i vanjske podatke (podaci o vremenu i najavljenim radovima u mreži) kao ulazne varijable. Ta metoda zajedno s istraženim proaktivnim metodama rane detekcije čini sustav inteligentnih metoda za predviđanje pojava smetnji u širokopojasnim mrežama.The aim of this thesis is to explore the opportunities of short-and long-term predictions of the expected number of failures in broadband telecommunications networks. Selection of the optimal prediction method depends on the nature of the processes being modeled, data availability, and the duration of the monitoring period, as well as on adaptability of involved operational support systems. Testing is conducted on actual telecommunications network data collected during the period between 2009-2012. The dataset consisted of over 2,5 million measured values. The research focus has been on dynamic, recursive neural networks specialized for prediction in nonlinear systems, but for the purpose of comparison, traditional autoregressive prediction methods are considered too. A number of parameters have been adjusted to obtain the maximum accuracy of each individual method and to select the most efficient predictive method for specific test cases. We have shown that recursive neural network is flexible enough to approximate the dynamics of the failure reporting process. Many factors, both in the network and outside the network, influence the time series representing failure reporting. The model encompasses the most important predictor variables and their logical and temporal dependencies. Predictor variables represent internal factors such as profiles of past and current quantities of failures as well as external factors like weather forecasts or announced activities (scheduled maintenance) in the network. External factors have a strong effect on fault occurrence, which finally results in failures reported by users. These factors are quantified and included as input variables to our model. The model is fitted to the data from different sources like an error-logging database, a troubleticket archive, announced settings logs and a meteo-data archive. The accuracy of the model is examined on simulation tests varying the prediction horizons. Assessment of the model’s accuracy is made by comparing results obtained by prediction and the actual data. The developed prediction model is scalable and adaptable so that other relevant input factors can be added as needed. Prediction method based on recursive neural network model together with designed proactive methods for early fault detection represent a system of intelligent methods for failure prediction in broadband networks. This thesis is organized as follows. In the introductory Section the objectives and motivation for the research are described. The basic terms and concepts are also defined. Section 2 ("2. Failures in broadband telecommunications networks") introduces architecture of broadband network. Locations and causes of faults and failures in a network are explained. Furthermore, fault repair process and system environment with integrated functional blocks are presented in Section 3 ("Faiure repair process"). TMN and eTOM standard models are briefly described too. In the fourth Section ("4. Diagnostic procedures and their improvements") procedures for the improvement of diagnostics related to an access part of the network are presented. Procedures rely on factor analysis and Bayesian networks and increase the quality of the model and the accuracy of diagnosis. The fifth Section ("5. Methods for early detection that enable proactive actions in the access network") presents two concrete solutions. The first one is a system for early detection of problematic installations. The second one is a model for the early detection of copper pairs that are poorly protected from moisture and therefore they are subject to frequent failures. Section 6 (" 6. Distribution of the time that elapses from the occurrence of alarms to the moment when customers report failures ") introduces the data set used and characteristics of the failure reporting behavior. To exploit characteristics of the customers’ failure reporting behavior it is important to quantify the speed of customers' reaction to the fault occurrences and service failures. Customers' failure reporting behavior is complex. The time that elapses from element fault that causes service failure to the moment of reporting the failure dominantly depends on firstly customers' average daily usage of services (whether the customer is using the service at the time or shortly after the outage occurs) and secondly customers’ expected actions/behavior (active or passive/indifference) in the moment when he/she becomes aware of service failure. We approximated the empirical data with the following four distributions: Weibull, Poisson, Gamma and Lognormal, conclusion was that Lognormal distribution represents the set of empirical data well. Time series representing dynamics of the failure reporting process are presented in Section 7 ("7. Choice of the optimal method for predicting quantity of reported failures"). A multivariate model for short-time prediction of reported failure quantities based on the recursive neural network, together with an approach to determination of significant predictor variables, is described. Accuracy of the model and predictions are presented in relation to variable prediction horizons and the number of input variables. The Section ("8. Extension of the fault management system") shows how the models and methods for early fault detection as well as methods for predicting expected number of reported failures can be implemented in the existing fault management system. Methods and their efficiency are experimentally tested and verified in the operator's environment. Full integration of these methods in production environment requires comprehensive organizational and informational preparation and significant costs. Finally, the concluding Section 9 provides the summary of the work and short description of the achieved results