Resource Management for Enterprise Data Center Applications

Today\u27s enterprise depend heavily on their information technology (IT) infrastructure for performing a variety of tasks. The IT infrastructure of a large enterprise comprises of a large number of software applications that utilize various resources including both software and hardware resources. The massive scale of these enterprise applications as well the complicated interactions between its various resources, makes resources management of such enterprise applications extremely challenging. Also, there are dynamics at various time scales that need to be managed effectively. In this thesis we investigate various challenges in resource management of such large-scale distributed enterprise applications and develop systems that solve some of these management tasks arising at multiple time scales. First, we study the problem of modeling and analysis in complex enterprise applications for predicting the impact of workload changes. Administrators often need to perform what-if analysis to manage enterprise applications and handle long-term changes in incoming workload. We discuss Predico, a system that automatically builds models of the enterprise application by first creating models of its components and then building large models by combing these smaller models. We then employ algorithmic techniques to use these models and enable administrators to perform workload-based what-if exploration of the application. We then discuss the problem of IT transformation and consolidation. Large enterprise often perform the task of enterprise-wide IT consolidation to simplify their IT infrastructure and reduce operational costs by consolidating their servers into fewer data centers. This is a very challenging exercise since it involves simultaneously optimizing multiple objectives like application performance, cost, resiliency to disasters, dependencies between application components, etc. While currently this extremely complicated task is being performed through simple techniques, we propose a system called eTransform that uses an optimization-based approach to automatically generates IT consolidation plans. Next, we study the problem of automatically and dynamically allocating resources for online web applications to react to the changing nature of workload while also meeting service level agreements. We present a dynamic resource provisioning scheme that automatically categorizes the workload into different classes based on the service demand and then intelligently provisions resources based on the changing mix of workload. We implement and evaluate our provisioning scheme on both private and public cloud platforms where it is able to effectively avoid performance degradation and reduce resource over-allocation. Finally, we present Yank, a system for handling transiency in enterprise data centers. Transiency arises in various scenarios in today\u27s data centers. For example, green data centers that use renewable energy sources are exposed to transiency due to the intermittent nature of such power sources. We introduce the abstraction of transient servers to handle transiency and then provide system support for transient servers through a system called Yank. We illustrate how Yank enables green data centers to use a mix of grid power and renewable power while hiding the transiency of renewable-powered servers from the applications