Critical database technologies for high energy physics

A number of large-scale high energy physics experiments loom on the horizon, several of which will generate many petabytes of scientific data annually. A variety of exploratory projects are underway within the physics computing community to investigate approaches to managing the data. There are conflicting views of this massive data problem: (1) there is far too much data to manage effectively within a genuine database; (2) there is far too much data to manage effectively without a genuine database; and many people hold both views. The purpose of this paper is to begin a dialog between the computational physics and very large database community on such problems, and to simulate research in directions that will be of benefit to both groups. This paper will attempt to outline the nature and scope of these massive data problems, survey several of the approaches being explored by the physics community, and suggest areas in which high energy physicists hope to look to the database community for assistance