Peer-to-Peer Architectures for Data Discovery, Distribution and Consistent Replication

Despite the push towards consolidation of both data and computational resources into increasingly larger data centers, a majority of companies and organizations still rely on multiple small server rooms whose sizes do not exceed 5000 sq. ft. Our research proposes a Peer-to-Peer based architecture that provides two of the major services offered by consolidated data center systems, namely rapid deployment of large virtual machine (VM) images and applications and high-performance distributed storage. Our VM dissemination approach uses a new multi-layered design combined with a two-stage query mechanism. These enable the publishing and querying of dynamically-changing VM information, thereby reducing the deployment time of virtual machines and applications to clients. The opportunistic replication of VM data afforded by such dissemination mechanisms was further coupled with the replicated transactional mechanisms demonstrated within our Peer-to-Peer (P2P) based storage scheme. Such combined systems provide the deployed virtual machines and applications with a fault-tolerant, high-performance computational space upon which to concurrently store and retrieve large volumes of data in a mutually consistent manner. Our deployment architecture has been evaluated against existing VM-dissemination mechanisms and demonstrated significant improvements in VM deployment time, with at least an 8% improvement over existing high-performance content-distribution designs. Similarly, our data-storage architecture improves the performance of established computational benchmarks by at least 22% over existing replicated storage mechanisms. Our developed approaches were also able to facilitate these improvements without a corresponding major increase in traffic overhead, even as the size of the evaluated systems increased. This demonstrated the scalability of our designs and their suitability for use in connecting large numbers of widely distributed data centers