A Benchmark Suite for Distributed Publish/Subscribe Systems

Building a distributed publish/subscribe infrastructure amounts to dening a service model (or in-terface) and providing an implementation for it. A typical distributed implementation is architected as a network of dispatcher components, each one implementing appropriate protocols and algorithms, that collectively realize the chosen service model. The service model should provide a value-added service for a wide variety of applications, while the implementation should gracefully scale up to handle an intense trafc of publications and subscriptions. We believe that the design of such service models and implemen-tations must be guided by a systematic evaluation method, which in turns must be based on a carefully chosen benchmark suite. In this paper, we lay out a set of requirements for a benchmark suite for distributed publish/subscribe services, and we outline its primary components. The ideas proposed in this paper are based on our own experience in building and studying publish/subscribe infrastructures, and on existing evaluation frameworks