The production cluster construction checklist.

This document is a detailed checklist of the steps that one must go through to bring up a production computing cluster. The list starts with planning activities and culminates in the activities necessary to operate and sustain a production computing facility. This checklist is derived from a number of experiences installing real-world, large-scale clusters. While each installation experience was unique, we were interested in determining the common characteristics across each deployment. We collected all of the to-do lists, presentations, notes, email messages, white board notes, and any other planning tools we could find from each of the installation activities. We combined them into a huge, messy diagram that was probably impossible to understand without having been involved in its creation but was excellent for identifying differences and commonalities. After organizing, checking, and distilling the information, we created the checklist presented here. Interesting is the fact that the high-level activities on the resulting list are neither cluster nor computer specific. Most of these activities would be followed when installing a production computer of any architecture or when installing any kind of complex facility that will eventually support users. The purpose of this list is not to give step-by-step instructions but rather to serve as a guide and a reminder. The items on the list are necessarily brief statements. Detailed explanations of these would go beyond the intended scope of the list. The list is organized in outline fashion. The major phases of construction are individual sections. Each of the subsections is a task or subtask in that phase. The items on this list are presented in a logical sequence, in approximately the order that one would follow if one were to start with a budget and an idea. However, every cluster is different, and every situation for using clusters is different. Most likely, no one would ever follow the steps here in this exact order; many things can be done in a different order, simultaneously, or skipped altogether. The list, for example, may place more emphasis on testing than many sites formally will