An Evaluation of State Sharing Techniques in Distributed Operating Systems

A shared memory abstraction in distributed systems (DSM) provides ease of programming but could be costly to implement. Many protocols have been proposed recently that are based on different approaches for exploiting program semantics. We have implemented four different protocols that embody the different memory semantics and have evaluated them using applications that capture a wide range of state sharing patterns. Our main goal is to quantify the relative performance of these protocols with respect to representative applications. We expect the results of such a study to be useful for an operating system designer to choose the right protocol to support the DSM abstraction. One of the surprising results is that memory systems that provide weaker consistency or use synchronization information in coherence maintenance (e.g. release consistency) do not provide significantly better performance than the basic invalidation based protocol. The disparity between the performance using these protocols and programmer controlled state sharing can range from being insignificant to as high as 50% depending on application characteristics. The use of synchronization information in conjunction with user directives will bridge this performance gap but may decrease the programming ease of DSM