On Measuring the Internet with a Mobile Crowdsourcing System

Given the commercial nature of the Internet, knowledge on its properties and structure is not publicly available, and can be obtained only through measurements. However, measuring such a tangled system is a huge task that can be overwhelming for a single organization. In this PhD thesis we tackled the problem with a crowdsourcing approach. We present Portolan, a crowdsourcing system based on mobile devices, able to measure multiple properties of the Internet. This work focuses on the server infrastructure of Portolan. The architecture of the system has been designed to be modular and scalable. New measurement subsystems can be added with small effort and the system is able to handle a large number of measuring clients. As case studies, we focus on two measurement subsystems. The first is aimed at discovering the Internet topology at the Autonomous System (AS) level of abstraction. For the first time, to perform such task, devices with scarce resources enrolled via crowdsourcing have been used. Taking into account the devices' limitations, we designed a measurement methodology focused on keeping a low measurement load on the system. We analysed one year of measurements to show that the designed methodology is able to discover hidden parts of the Internet without consuming the scarce resources of the measuring agents. The second measurement subsystem is aimed at geolocalizing Internet hosts with delay measurements. We present an IP geolocation method designed to cope with measurements that, differently from existing literature, are run in a wireless scenario. We show that crowdsourcing allows to attenuate the noise of measurements and significantly reduce the geolocation error. These positive experimental results obtained with Portolan in both cases confirm the soundness of the crowdsourcing approach based on mobile devices