Appraising and improving the accuracy of smartphone and browser-based delay measurement

PolyU Library Call No.: [THS] LG51 .H577P COMP 2017 Lixix, 165 pages :color illustrationsIn recent years, using network performance bench marking tools, such as Ookla Speedtest, to understand network performance has become popular. These measurements are available in desktop environment via Web browsers (browser-based measurements) and mobile environment via mobile apps (smartphone-based measurements), which provides a very effective approach to crowdsourcing network performance data. However, despite their popularity, very little is known about the (in)accuracy of the various methods used by these tools. In this research, we contribute to examining and improving the accuracy of these tools. We focus on the network delay measurement, because it is the most fundamental and atomic metric. To quantify the inaccuracy, we define "delay overhead" as the difference between the value measured by the measurement tools and the actual network delay. The delay overhead, which is considered as measurement noise, should be avoided and mitigated during delay measurements to achieve more accurate results. In browser-based measurement, we evaluate the accuracy of twelve methods, including the traditional HTTP-based and TCP socket-based methods, Navigation Timing and WebRTC techniques, with the eight most popular browsers on Linux and Windows. Our evaluation results show that the delay overheads incurred in most of the HTTP-based methods are too large to ignore. Specifically, the overheads incurred by some methods(such as Flash GET and POST) vary significantly across different browsers and systems, making it very difficult to calibrate. The socket-based methods, on the other hand, incur much smaller overhead. Another interesting and important finding is that Date.getTime(), a typical timing API in Java, does not provide the millisecond resolution assumed by many measurement tools on some OSes. This results in a serious under-estimation of RTT. Moreover, some tools over-estimate the RTT by including the TCP hand shaking or data channel establishing phase.For the mobile network measurement, we show that the two most popular measurement appsOokla Speedtest and MobiPerfhave their RTT measurements inflated. We then build three corresponding test apps that cover three common measurement methods and evaluate them in a testbed. We overcome the main challenge of obtaining a complete trace of packets and their timestamps by using multiple sniffers and frame-based synchronization. Our multi-layer analysis reveals that the delay inflation can be introduced both in the user space and kernel space. The long path of subfunction invocations accounts for the majority of the delay overhead in Android runtime (both Dalvik VM and ART), and the sleeping functions in the drivers are the major source of the delay overhead between the kernel and physical layer. Based on our evaluation in smartphones, we try to reduce and stabilize the delay inside the phone as much as possible. We report for the first time a major source of noise comes from the periodical SDIO (Secure Digital Input Output) bus sleep inside the phone. Besides, the PSM (Power Saving Mode) for WiFi networks and the RRC (Radio Resource Control) states in cellular networks will also unstably inflate the delay measurement. To mitigate these measurement noises, we propose to keep the phone in the wake-up mode or high-power state during the delay measurement by sending just a sufficient amount of warm-up and background traffic. We implement this approach in AcuteMon, an Android app, and validate it in testbed and in real users' phones. The evaluation shows that AcuteMon can effectively mitigate the delay overheads caused by various energy-saving mechanisms, and obtain more accurate network delay.Department of ComputingPh.D., Department of Computing, The Hong Kong Polytechnic University, 2017Doctorat