Distributed Tracking, Storage, and Re-use of Job State Information on the Grid

The Logging and Bookkeeping service tracks job passing through the Grid. It col-lects important events generated by both the grid middleware components and ap-plications, and processes them at a chosen L&B server to provide the job state. The events are transported through secure reliable channels. Job tracking is fully distributed and does not depend on a single information source, the robustness is achieved through speculative job state computation in case of reordered, delayed or lost events. The state computation is easily adaptable to modified job control flow. The events are also passed to the related Job Provenance (JP) service. Its purpose is a long-term storage of information on job execution, environment, and the exe-cutable and input sandbox files. The data can be used for debugging, post-mortem analysis, or re-running jobs. The data are kept by the job-provenance storage service in a compressed format, accessible on per-job basis. A complementary index service is able to find particular jobs according to configurable criteria, e.g. submission time or "tags " assigned by the user