Analyzing the impact of local perturbations of network topologies at the application-level

Networked systems are continuously growing in scale and complexity. The technical and policy engineering challenges introduced by such a fast growth are currently addressed locally, with limited understanding of their impact on the whole. Such approaches are becoming impractical and insufficient. Next-generation networks need to address these issues by deploying adaptive and self-managing protocols and mechanisms to relax the persistent need for human-driven management. However, achieving these objectives requires conceptual, physical, and logistical modifications to existing systems and protocols. To this end, the traditional top-down approach to network and application design needs to be supplemented by understanding the bottom-up nature of evolving real-world networks. A critical issue that is significantly impacting computer networks and applications is the absence of an in-depth understanding and lack of control over the structural properties, i.e., topology, of large networks. Network topologies define the link relationships between the nodes in the network, and have a direct impact on the performance, resilience, and security of distributed applications. Large scale networks such as the Internet are the result of a time evolving process in which nodes and links between nodes are added, removed, and reconfigured dynamically. This dynamic process takes place in a decentralized manner during which nodes make local adaptations and reconfiguration decisions that optimize local properties. As a result, these local perturbations yield an emergent network that is often unstructured and complex, and have implications at the application-level, particularly impacting routing, search, robustness, and clustering. Understanding the structures emerging out of these adaptations is a complex problem part of the science and study of complexity theory and complex adaptive systems. Tackling this complex problem requires first, identifying canonical metrics to quantify the network topology and second, analyzing the impact of local perturbations of these metrics on the resulting network topology. This thesis identifies three local metrics, transitivity, assortativity, and entropy, and analyzes the impact of their perturbation on the applications of routing, search, robustness, and clustering. The local metric of network entropy is identified as a useful information theoretic measure of homogeneity of a network neighborhood degree. The metric is further used to derive a novel mechanism of clustering detection of the network topology. The overall objective of this thesis is to investigate metrics and mechanisms to better understand the evolution of the network topology and its impact on application-level functionality. The approach is based on concepts of emergence, self-organization and graph theory, and has three key aspects: (1) the identification of canonical local and global graph metrics; (2) the quantitative analysis of the impact of local perturbations on global properties; and (3) the application of the local to global mapping on the problems of routing, search, robustness, and clustering. Adaptations are performed in a decentralized manner in which local nodes use local information to add, remove, or rewire an edge to evolve the topology. Simulations based on annealing optimization are conducted to empirically determine the optimal bounds of the network structures for the selected metrics on selected networks. Further experiments on two modeled networks, random and power-law degree distributed, and two real-world networks, the Gnutella and Canadian Autonomous System networks, show that the impact of optimizing networks with fixed degree distribution on local metrics yield networks with routing, search, robustness, and clustering that are tightly dependent on the network's degree distribution. A key outcome of this thesis is the identification of network entropy minimization as a useful local rewiring strategy to decrease average path length and search cost, while homogenizing the size of network clusters and having a low impact on robustness when applied to power-law degree distributed networks that prevail in real-world networks.Ph.D.Includes bibliographical references (p. 140-144)