HPC-Colony: services and interfaces for very large systems

Traditional full-featured operating systems are known to have properties that limit the scalability of distributed mem-ory parallel programs, the most common programming para-digm utilized in high end computing. Furthermore, as pro-cessor counts increase with the most capable systems, the necessary activity to manage the system becomes more of a burden. To make a general purpose operating system scale to such levels, new technology is required for parallel resource management and global system management (in-cluding fault management). In this paper, we describe the shortcomings of full-featured operating systems and runtime systems and discuss an approach to scale such systems to one hundred thousand processors with both scalable parallel ap