Cluster computing on the fly: P2p scheduling of idle cycles in the internet

Abstract — Peer-to-peer computing, the harnessing of idle compute cycles throughout the Internet, offers exciting new research challenges in the converging domains of networking and distributed computing. Our system, Cluster Computing on the Fly, seeks to harvest cycles from ordinary users in an open access, non-institutional environment. We identify four important classes of cycle-sharing applications, each with distinct requirements that call for application-specific scheduling strategies. Our Wave Scheduler exploits large blocks of idle time at night, to provide higher quality of service for deadline-driven workpile jobs, using a geographic-based overlay to organize hosts by timezone. To verify the results of workpile jobs, CCOF sends quizzes to hosts and uses the accuracy of the quiz answers to determine trust ratings. Our PoP Scheduler disperses tasks comprising a point-ofpresence application, using scalable protocols to discover strategically located hosts to meet application-specific requirements for location, topological distribution, and resources. The CCOF cycle sharing system encompasses all activities involved in the management of idle cycles: overlay construction for hosts donating cycles, resource discovery within the overlay, application-based scheduling, local scheduling on the host node, and meta-level scheduling among a community of application-level schedulers. Our work with CCOF reveals many of the critical challenges that lie ahead for P2P scheduling systems. I