P2P Storage Systems: How Much Locality Can They Tolerate?

Large scale peer-to-peer systems are foreseen as a way to provide highly reliable data storage at low cost. To achieve high durability, such P2P systems encode the user data in a set of redundant fragments and distribute them among the peers. We study here the impact of different data placement strategies on the system performance when using erasure codes redundancy schemes. Several practical factors (easier control, software reuse, latency) tend to favor data placement strategies that preserve some degree of locality. In this paper, we compare three policies: two of them local, in which the data are stored in logical neighbors, and the other one global, in which the data are spread randomly in the whole system. We focus on the study of the probability to lose a data block and the bandwidth consumption to maintain such redundancy. We use simulations to show that, without resource constraints, the average values are the same no matter which placement policy is used. However, the variations in the use of bandwidth are much more bursty under the local policies. When the bandwidth is limited, these bursty variations induce longer maintenance time and henceforth a higher risk of data loss. We then show that a suitable degree of locality could be introduced in order to combine the efficiency of the global policy with the practical advantages of a local placement. Finally, we propose a new external reconstruction strategy that greatly improves the performance of local placement strategies