Autonomous Replication for High Availability in Unstructured P2P Systems

We consider the problem of increasing the availability of shared data in peer-to-peer (P2P) systems so that users can access any content, regardless of the current subset of online peers. In particular, we seek to conservatively estimate the amount of excess storage required to achieve a practical availability of 99.9% by studying a decentralized algorithm that only depends on a modest amount of loosely synchronized global state. Our algorithm uses randomized decisions extensively together with a novel application of the Reed Solomon erasure codes to tolerate autonomous member actions as well as staleness in the loosely synchronized global state. We study the behavior of this algorithm under three distinct environments, modeled using data published from earlier studies of a corporate environment, the Napster and Gnutella file sharing communities, and a file sharing community local to students of our University. We show that while peers act autonomously, the community as a whole will reach a relatively stable configuration. We also show that space is used fairly and efficiently, delivering three nines availability at a cost of six times the storage footprint of the data collection when the average peer availability is only 24%.Technical report DCS-TR-50