Job Scheduling in Grid Farms

The vast majority of jobs in the Grid are embarrassingly parallel. Inparticular HEP tasks are divided into atomic jobs without need forcommunication between them. Jobs are still neither multi-threaded normulti-core capable. On the other hand, resource requirements reachfrom CPU-dominated Monte Carlo jobs to network intense analysis jobs.The main objective of any Grid site is to stably operate its Grid farmwhile achieving a high job slot occupancy, an optimal usage of thecomputing resources (network, CPU, memory, disk space) and guaranteedshares for the VOs and groups. In order to optimize the utilization ofresources, jobs must be distributed intelligently over the slots,CPUs, and hosts. Although the jobs resource requirements cannot bededuced directly, jobs are mapped to POSIX user/group ID based ontheir VOMS-proxy. The user/group ID allows to distinguish jobs,assuming VOs make use of the VOMS group and role mechanism.The multi-VO Tier-2 site at DESY (DESY-HH) supports ~20 VOs onfederated computing resources, using an opportunistic resource usagemodel. As at many EGI/WLCG sites, the Grid farm is based on thequeuing system PBS/TORQUE, which was deployed from the EMI middlewarerepositories. Initially, the scheduler MAUI was used. It showed severescalability problems with 4000 job slots as soon as the number ofrunning plus queued jobs approached 10000. Job scheduling became slowor even blocked. In addition, MAUI's many configuration optionsappeared to be hard to control.To be able to further increase the number of worker nodes asrequested by the VOs (to currently 8000 job slots), DESY-HH needed ascalable and performing scheduler, which runs in conjunction withPBS/TORQUE. In the course of studying alternative scheduling models,a home-made scheduler was developed (working title: MySched), which istailored to then needs of the DESY-HH Grid farm and uses the C-API ofPBS/TORQUE. It is based on a simple scheduling model without supportfor multi-core jobs and job parallelism and is optimized for highjob slot occupancy and intelligent distribution of jobs to the workernodes. Furthermore, it allows for a fine-grained adjustment oflimits and parameters on VO and group level.In the contribution to CHEP 2013 we will discuss the impact of aclassification of jobs according to their potential resourcesrequirements on scheduling strategies. Subsequently, we will describeour home-made implementation and present operational results