A design framework for highly concurrent systems

Building highly concurrent systems, such as large-scale Internet services, requires managing many information ows at once and maintaining peak throughput when demand ex-ceeds resource availability. In addition, any platform sup-porting Internet services must provide high availability and be able to cope with burstiness of load. Many approaches to building concurrent systems have been proposed, which generally fall into the two categories of threaded and event-driven programming. We propose that threads and events are actually on the ends of a design spectrum, and that the best implementation strategy for these applications is somewhere in between. We present a general-purpose design framework for build-ing highly concurrent systems, based on three design com-ponents | tasks, queues, and thread pools | which encapsu-late the concurrency, performance, fault isolation, and soft-ware engineering benets of both threads and events. We present a set of design patterns that can be applied to map an application onto an implementation using these compo-nents. In addition, we provide an analysis of several systems (including an Internet services platform and a highly avail-able, distributed, persistent data store) constructed using our framework, demonstrating its benet for building and reasoning about concurrent applications.