System Management in the BlueGene/L Supercomputer

The BlueGene/L supercomputer will use system-on-a-chip integration and a highly scalable cellular architecture to deliver 360 Teraflops of peak computing power. With 65,536 compute nodes, BlueGene/L represents a new level of scalability for parallel systems. As such, it is natural for many scalability challenges to arise. In this paper, we discuss challenges in the area of system management and control, including machine booting, software installation, user account management, system monitoring, and job exe-cution. We address the issue of scalability by organizing the system hierarchically. The 65,536 compute nodes are or-ganized in 1,024 clusters of 64 compute nodes each, called processing sets. Each processing set is under control of a 65th node, called an I/O node. The 1,024 processing sets can then be managed to a great extent as a regular Linux cluster, of which there are several successful examples. Reg-ular cluster management is complemented by BlueGene/L specific services, performed by a service node over a sep-arate control network. Our software development and ex-periments have been conducted so far in architecturally ac-curate simulators of BlueGene/L, and we are gearing up to test real prototypes in 2003.