Integrating Remote Invocation and Distributed Shared State

Most distributed applications require, at least conceptually, some sort of shared state: information that is non-static but mostly read, and needed at more than one site. At the same time, RPC-based systems such as Sun RPC, Java RMI, CORBA, and .NET have become the de facto standards by which distributed applications communicate. As a result, shared state tends to be implemented either through the redundant transmission of deep-copy RPC parameters or through ad-hoc, application-specific caching and coherence protocols. The former option can waste large amounts of bandwidth; the latter significantly complicates program design and maintenance. To overcome these problems, we propose a distributed middleware system that works seamlessly with RPC-based systems, providing them with a global, persistent store that can be accessed using ordinary reads and writes. In an RPC-based program, shared state serves to (1) support genuine reference parameters in RPC calls, eliminating the need to pass large structures repeatedly by value, or to recursively expand pointer-rich data structures using deep-copy parameter modes; (2) eliminate invocations devoted to maintaining the coherence and consistency of cached data; (3) reduce the number of trivial invocations used simply to put or get data. Relaxed coherence models and aggressive protocol optimizations reduce the bandwidth required to maintain shared state. Integrated support for transactions allows a chain of RPC calls to update that state atomically. We focus in this paper on the implementation challenges involved in combining RPC with shared state, relaxed coherence, and transactions. In particular, we describe a transaction metadata table that allows processes inside a transaction to share data invisible to other processes and to exchange data modifications efficiently. Using microbenchmark and large-scale datamining applications, we demonstrate how the integration of RPC, transactions, and shared state facilitates the rapid development of robust, maintainable code