Graph-based decision making for task scheduling in concurrent Gaudi

The modern trend of extensive levels of hardware parallelism and heterogeneity pushes software to evolve through a paradigm shift towards concurrent data processing architectures. One such striking example in the domain of high-energy physics is represented by Gaudi - an experiment independent software framework, used in two of four major experiments of the Large Hadron Collider project, and in several others. The framework is responsible for event processing by means of hundreds of algorithms which have logical and data dependencies between each other. Historically, the framework was designed being inherently sequential, meaning that at any time of data processing there is only one event being processed and only one algorithm being executed on it. This allowed to respect the dependencies of algorithms by just organizing them in a well-defined execution path to be run on CPU. The evolution of the Gaudi framework into its concurrent incarnation, though, implies the necessity to split the execution path dynamically into subsets of algorithms to fill up efficiently the available computing resources. In this work we present a graph-based decision making system as a solution to the problem. The approach allows to form and control dynamically the order of concurrent algorithms' execution, restricted by the topology of their dependencies of any complexity level. Furthermore, we show the system's capability of configuration- and run-time planning for optimal resource usage, and discuss a few concrete scheduling strategies, that this approach exposes