Network Performance: Does It Really Matter To Users and by How Much

Abstract—Network performance has been the subject of much research over the past decades. However, the impact of perfor-mance on a network’s users is much less understood from a scientific standpoint. This gap in our knowledge is particularly stark since the primary role of real-world network performance is to increase user satisfaction, and encourage user behaviors that lead to greater monetization. As an example, consider the video delivery ecosystem consisting of content delivery net-works (CDNs), video content providers, and their users. Content providers use CDNs to deliver videos to their users with higher performance. In turn, content providers expect the higher quality stream delivery to translate into lesser viewer abandonment, greater viewer engagement, and more repeat viewers, all of which lead to greater profits. Thus, whether and to what extent video performance causally impacts viewer behavior is at the core of the online video ecosystem. We review our prior research that for the first time derives the causal impact of video performance as measured by failures, startup delay and freezes on metrics of user behavior that content providers care about. A centerpiece of our work is a novel technique based on quasi-experimental designs (QEDs) that enable us to derive cause-effect relationships between performance and behavior with a greater degree of confidence than just correlation. While QEDs are used extensively in the social and medical sciences, we adapt the technique for network measurement research that is likely to be useful in a number of other contexts. We hypothesize that the availability of large amounts of performance and behavioral data and the develop-ment of novel analytical tools will finally put the users back at the center of network performance research, revolutionizing both how networks are architected for performance and how business models are evolved for real-world networks. I