Speedtest-like Measurements in 3G/4G Networks: the MONROE Experience

Mobile Broadband (MBB) Networks are evolving at a fast pace, with technology enhancements that promise drastic improvements in capacity, connectivity, coverage, i.e., better performance in general. But how to measure the actual performance of a MBB solution? In this paper, we present our experience in running the simplest of the performance test: ``speedtest-like'' measurements to estimate the download speed offered by actual 3G/4G networks. Despite their simplicity, download speed measurements in MBB networks are much more complex than in wired networks, because of additional factors (e.g., mobility of users, physical impairments, diversity in technology, operator settings, mobile terminals diversity, etc.). We exploit the MONROE open platform, with hundreds of multihomed nodes scattered in 4 different countries, and explicitly designed with the goal of providing hardware and software solutions to run large scale experiments in MBB networks. We analyze datasets collected in 4 countries, over 11 operators, from about 50 nodes, for more than 2 months. After designing the experiment and instrumenting both the clients and the servers with active and passive monitoring tools, we dig into collected data, and provide insight to highlight the complexity of running even a simple speedtest. Results show interesting facts, like the occasional presence of NAT, and of Performance Enhancing Proxies (PEP), and pinpoint the impact of different network configurations that further complicate the picture. Our results will hopefully contribute to the debate about performance assessment in MBB networks, and to the definition of much needed benchmarks for performance comparisons of 3G, 4G and soon of 5G networks