Having Your Cake and Eating It Too: Jointly Optimal Erasure Codes for I/O, Storage and Network-bandwidth

Erasure codes, such as Reed-Solomon (RS) codes, are increasingly being deployed as an alternative to data-replication for fault tolerance in distributed storage sys-tems. While RS codes provide significant savings in storage space, they can impose a huge burden on the I/O and network resources when reconstructing failed or otherwise unavailable data. A recent class of era-sure codes, called minimum-storage-regeneration (MSR) codes, has emerged as a superior alternative to the popu-lar RS codes, in that it minimizes network transfers dur-ing reconstruction while also being optimal with respect to storage and reliability. However, existing practical MSR codes do not address the increasingly important problem of I/O overhead incurred during reconstructions, and are, in general, inferior to RS codes in this regard. In this paper, we design erasure codes that are simultane-ously optimal in terms of I/O, storage, and network band-width. Our design builds on top of a class of powerful practical codes, called the product-matrix-MSR codes. Evaluations show that our proposed design results in a significant reduction the number of I/Os consumed dur-ing reconstructions (a 5 × reduction for typical parame-ters), while retaining optimality with respect to storage, reliability, and network bandwidth.