Characterizing and Optimizing Asynchronous Event-Driven Architecture for Modern Cloud Systems

Achieving good performance and high efficiency simultaneously is an essential requirement for emerging modern cloud systems such as e-commerce due to their business impact. For example, Akamai reported that every 100ms delay in website load time could lead to a 6% drop in sales. Unfortunately, achieving good performance (e.g., low latency) for modern cloud systems at high resource utilization is significantly challenging. Despite continuing efforts by cloud professionals, however, they have consistently experienced performance degradation problems (e.g., the long-tail latency problem) due to the bursty workload in the cloud. To resolve the performance degradation problems, many previous research efforts have advocated that the asynchronous event-driven architecture could be a superior alternative to the traditional thread-based design due to reduced multithreading overhead when facing a high concurrency workload. However, taking advantage of the asynchronous event-driven architecture is a non-trivial task due to the inherently obscured non-sequential control flow. In this dissertation, we show that building good performance and high efficiency servers for large-scale cloud systems using the asynchronous event-driven architecture requires careful design of the event processing flow and the ability to adapt to the runtime bursty workload and network conditions. Inefficient event processing flow of asynchronous event-driven servers can hurt the performance of large-scale cloud systems via causing (i) unexpected multithreading overhead (e.g., context switches), (ii) unnecessary syscalls (e.g., select and epoll), and (iii) resource (e.g., CPU) contention and performance interference. We found that the inefficient event processing flow can arise from a wide range of factors at different modern cloud systems, including traditional n-tier systems, large-scale datastore systems, and emerging microservices-based systems. By combining empirical experimental study and sophisticated analytical methods to generate and analyze fine-grained monitoring data, we can identify and study the inefficient event processing flow of asynchronous event-driven servers systematically