Bandwidth management technique for hierarchical storage in large-scale multimedia servers

Using magnetic disks as a cache for tertiary storage has been shown to be an effective way to address the high storage costs of large-scale multimedia servers. We investigate a technique for managing the bandwidth of such a hierarchical storage design. In existing schemes, all data items are treated equally; and the same I/O rate is used to load data from tertiary storage when they are requested. In our approach, different loading rates are used for data items with different characteristics. For frequently used items, we keep a large percentage of their data in the disk buffer and need to use only a small I/O rate to load the missing portions on demand. On the contrary a larger portion of less frequently used items is kept in the tertiary storage. To minimize their access latencies, larger I/O rates are used to load the missing parts when these items are needed. We formally prove that this approach is better than using the same loading rate for all data items. We also show simulation results to quantitatively demonstrate the benefits of our technique. They confirm that our scheme is able to provide higher system throughput while ensuring very short latencies (i.e., several seconds) for essentially all accesses. Such good performance is achieved using surprisingly small disk space (i.e., about 5% or less of the database size)