P oS(ACAT08)042 Job centric monitoring for ATLAS jobs in the LHC Computing Grid

large scale computing grid LCG (LHC Computing Grid) is meant to process and store the large amount of data created in simulating, measuring and analyzing of particle physic experimental data. Data acquired by ATLAS, one of the four big experiments at the LHC, are analyzed using compute jobs running on the grid and utilizing the ATLAS software framework ATHENA. The analysis algorithms themselves are written in C++ by the physicists using Athena and the ROOT toolkit. Identifying the reason for a job failure (or even the occurance of the failure itself) in this context is a tedious, repetitive and sometimes unfeasable task. Often, to deal with failures in the RUNNING stage (as opposed to job submission failures or compilation errors in the user algorithms), the job is just being resubmitted. In a grid context, users are never allowed to directly access the worker nodes (WNs) their jobs run on. So, debugging of a user job interactively is not possible. It is even more complicated because the output-sandbox, which contains the jobs ’ output and error logs, usually is discarded by the grid middleware if the job failed. So, valuable information that could aid in finding the reason of failure is lost. These issues result in high job failure rates and less than optimal resource usage