Storage Workload Characterization and Performance Prediction for Better I/O Traffic Management

In distributed high-performance computing storage systems, contention is a problem that usually occurs when the volume of client I/O requests exceeds the capability of storage servers; it degrades the whole system’s performance. The Storage Traffic Control System (STCS) is a critical component of a large system, which reduces the contention between clients, alleviates congestions in the system, and improves network and storage system efficiency. In a rule-based STCS, the building process for an optimal Storage Traffic Control Rule Set (STCRS) usually requires repetitious and lengthy benchmarking and tweaking cycles. Furthermore, most STCRSs are workload-specific and the established rule sets only matches to a limited subset of the actual workloads a storage system needs to handle. To add adaptiveness to the rule-based STCS, we present a performance prediction approach to help choosing existing STCRS for new workloads. This approach simplifies the traffic management for large-scale systems: instead of redesigning traffic control rules for each new workload the system receives, the proposed method extracts and measures the features of new workload and uses a regression tree pipeline to predict the performance improvement each control rule set bring to the new workload. The rule set that can best improve the performance of a workload will be chosen and applied to the new workload. Evaluation results show that most of the regression models we build have a high correlation coefficient (0.87) and low mean absolute error (3.8%). Using these performance prediction models, the suggested approach added adaptability to a rule based STCS by greatly increasing the amount of workloads that STCRSs can deal with