Improving load balance with flexibly assignable tasks

In many applications of parallel computing, distribution of the data unambiguously implies distribution of work among processors. But there are exceptions where some tasks can be assigned to one of several processors without altering the total volume of communication. In this paper, we study the problem of exploiting this flexibility in assignment of tasks to improve load balance. We first model the problem in terms of network flow and use combinatorial techniques for its solution. Our parametric search algorithms use maximum flow algorithms for probing on a candidate optimal solution value. We describe two algorithms to solve the assignment problem with \logW_T and vbar P vbar probe calls, w here W_T and vbar P vbar, respectively, denote the total workload and number of proce ssors. We also define augmenting paths and cuts for this problem, and show that anyalgorithm based on augmenting paths can be used to find an optimal solution for the task assignment problem. We then consider acontinuous version of the problem, and formulate it as a linearly constrained optimization problem, i.e., \min\|Ax\|_\infty,\; \rms.t. \; Bx=d. To avoid solving an intractable \infty-norm optimization problem, we show that in this case minimizing the 2-norm is sufficient to minimize the \infty-norm, which reduces the problem to the well-studied linearly-constrained least squares problem. The continuous version of the problem has the advantage of being easily amenable to parallelization. Our experiments with molecular dynamics and overlapped domain decomposition applications proved the effectiveness of our methods with significant improvements in load balance. We also discuss how our techniques can be enhanced for heterogeneous systems