Measuring, Understanding, and Improving Content Distribution Technologies

The Internet plays a focal role in our social life, work, education, and entertainment. The current Internet has been designed, over 50 years ago, for reliable host-to-host communications. Today, however, the Internet is mainly used for content distribution and retrieval. To cope with this shift in the Internet usage, three technologies for efficient content distribution were developed in the past years: Peer-to-Peer (P2P) systems, Content Delivery Networks (CDNs), and Information-Centric Networking (ICN). P2P systems and CDNs are widely used today, both constructing overlays atop the current Internet. ICN, in contrast, is a promising research direction proposing to redesign the Internet from scratch as a content-centric network. We focus on P2P systems and ICN. Despite the wide deployment of P2P systems and the promising features of ICN, several research problems in these technologies are still awaiting for effective solutions. We address in this dissertation three of them. We first deal with Kademlia, the most popular P2P system. Improving and giving guarantees on Kademlia's performance and robustness is needed to achieve a reliable and high quality service. This requires accurate understanding of system-wide topological properties such as the degree distribution and graph resilience, as well as properties of routing information. In spite of the importance of these properties, they were analysed so far only through partial measurements or oversimplified simulations. Instead, accurate results can be derived only from graph snapshots of real systems. However, capturing such snapshots is challenging since it requires to collect large amount of highly dynamic information, almost instantly, which was not provided by previous measurements. Results of such graph snapshots also can be used to develop accurate simulation models. We address this challenge by KadSpider, a crawler that we develop to capture highly representative graph snapshots of KAD (a popular implementation of Kademlia). Exploiting KAD design, KadSpider downloads the target information several times faster than prior Kademlia crawlers with lower signalling overhead. Analysing the captured snapshots and comparing them to synthetic graphs generated by simulations, we provide important information about KAD. We also show that, for certain properties, simulative results can significantly deviate from the results of the real system. Most interestingly, the complete graph in the real system, due to a greatly increased ratio of stale routing information, is much more vulnerable to targeted attacks than in simulations. This observation holds even when we simulate with a widely accepted churn model. Using the captured graph snapshots, we analyse the diversity of node's neighbours over the corresponding identifier space sections. We also show that the average routing hop count can be reduced when this diversity is maximized. Consequently, we apply this strategy in a modified neighbour selection scheme. The scheme is backward compatible with Kademlia and its derivatives, and incurs only negligible computational overhead. We demonstrate the utility of the scheme via theoretical model derivations, simulations of three notable Kademlia-type systems, and measurements on KAD nodes. After that, we shift our attention to Named-Data Networking (NDN), widely considered a promising ICN architecture for the future Internet. In particular, we address two salient problems in NDN: (i) cache management and (ii) defending against an NDN-tailored DDoS attack called the Interest Flooding Attack (IFA). We argue that both problems should be addressed in a coordinated way based on timely and network-wide content access information and other system states, which was not achieved in the past. The distributed nature of the Internet, in addition to the huge volumes and high dynamics of coordination information, however, make realizing such coordination challenging. We address this challenge by CoMon, a novel framework for Coordination in NDN that is based on (i) lightweight Monitoring of content access and resource usage and (ii) bounded advertisement of relevant information. CoMon assigns monitoring tasks to a small number of monitoring nodes selected such that the entire network traffic passes, or is enforced to pass, through them at an early stage. The monitoring nodes, in cooperation with a centralized controller, are also responsible for re-routing user requests towards in-network caches as well as for defending against IFAs. Our evaluations, based on extensive simulations, show that CoMon incurs negligible signalling overhead. They also show that CoMon enables to achieve both a remarkable caching efficiency improvement and high robustness against IFAs, in comparison both to the original system as well as to the state of the art