Turning Flash Crowds into Smart Mobs with Real-Time Stochastic Detection and Adaptive Cooperative Caching

A flash crowd occurs when a stampede of clients attempt to access a resource, e.g., a Web page. It can cause drastic performance degradation at servers and clients, and even outage of the service. In this paper, we first use traces collected from six different flash crowds to characterize the properties of flash crowds, showing in the process that ideal caching schemes can in fact mask flash crowds. We then present a stochastic detector that can alert a server about a flash crowd while the event is happening, i.e., in real-time. This scheme is then combined with cooperative caching among the clients to design and implement a system that adaptively detects and masks the effect of flash crowds. This mechanism effectively turns the clients in a flash crowd event into a ``smart mob''. We make our study comprehensive by comparing the performance of the above design experimentally with representative systems from three classes of existing designs for content providing - single server, server cooperation, and server-client modification