Self -managing techniques for storage resource management

The increasing reliance on online information in our daily lives had called for a rethinking of how people manage and maintain computer systems. As information has become more valuable and computing environments more complex, improved manageability has become key to ensuring availability. The sheer size of enterprise-scale storage systems coupled with the diversity and variability of application workloads makes their management non-trivial. Not surprisingly, numerous studies have shown that management costs have become a significant fraction of the total cost of ownership of large storage systems. Traditionally storage management tasks have been performed manually by administrators using a combination of experience, rules of thumb and trial and error. This increases the chance of a misconfigured or sub-optimally configured system. The cost of such misconfigurations can be high, since even a short downtime can result in substantial revenue losses. So, although storage is cheap, storage management is costly and storage mismanagement costlier. This argues the need for an automated, seamless and intelligent way to manage the storage resource. In this thesis, I propose self-managing techniques, specifically for resource management, to improve the manageability of large-scale storage systems. I have focused on techniques for automating two common storage allocation tasks: storage bandwidth allocation and storage space allocation. Large scale storage systems host data objects of multiple types which are accessed by applications with diverse service requirements. I have developed an online measurement based technique as well as one based on learning to dynamically partition bandwidth between application classes. Storage allocation algorithms that determine object placement, and thus the performance, are crucial to the success of a storage system. For a self-managing storage system a suitable placement technique is one that has low management overhead and delivers agreeable performance. In this context, I empirically compare different placement techniques to determine their suitability for large-scale storage systems, Finally, I also present techniques to minimize the amount of data displaced when remapping objects to eliminate hotspots