Scalable classification of QoS for real-time interactive applications from IP traffic measurements

Measurement of network Quality of Service (QoS) has attracted considerable research effort over the last two decades. The recent trend towards Internet Service Providers (ISP's) offering application-specific QoS is creating possibilities for more sophisticated QoS metrics to be offered by ISP's in service level agreements. This in turn could be used for the purposes of improved network optimization and billing according to application specific QoS guarantees. We report a scalable near real-time approach using passively logging IP traffic data for classification of application latency and packet loss across a range of real-time interactive applications. We run six experiments involving Minecraft, Quake 3 Urban Terror, VLC video streaming and the commercial Wirofon VOIP application. We use a mixture of laboratory and real-world deployments, with run times ranging from hours to days, and observe a combination of real and simulated ISP latency and packet loss events. Our binary classification (i.e. classes 'OK' or 'lag') 10-fold cross validation F1 scores are between 0.80 and 0.93 depending on application type. Our multi-class classification (i.e. classes representing discrete packet loss or latency ranges) 10-fold cross validation F1 scores for Minecraft are 0.89 for latency and 0.90 for packet loss. With new business models between ISP's and application developers being actively considered this work represents a significant contribution to the debate by providing scientific evidence relating to a novel approach to scalable QoS measuremen