Chameleon: a self-evolving, fully-adaptive resource arbitrator for storage systems

Enterprise applications typically depend on guaranteed performance from the storage subsystem, lest they fail. However, unregulated competition is unlikely to result in a fair, predictable apportioning of resources. Given that widespread access protocols and scheduling policies are largely best-effort, the problem of providing performance guarantees on a shared system is a very difficult one. Clients typically lack accurate information on the stor-age system’s capabilities and on the access patterns of the workloads using it, thereby compounding the problem. CHAMELEON is an adaptive arbitrator for shared storage resources; it relies on a combination of self-refining mod-els and constrained optimization to provide performance guarantees to clients. This process depends on minimal information from clients, and is fully adaptive; decisions are based on device and workload models automatically in-ferred, and continuously refined, at run-time. Corrective actions taken by CHAMELEON are only as radical as war-ranted by the current degree of knowledge about the sys-tem’s behavior. In our experiments on a real storage sys-tem CHAMELEON identified, analyzed, and corrected per-formance violations in 3-14 minutes—which compares very favorably with the time a human administrator would have needed. Our learning-based paradigm is a most promis-ing way of deploying large-scale storage systems that ser-vice variable workloads on an ever-changing mix of device types.