Scheduling Data and Task on Data-Driven Master/Worker Platform

International audienceWith data intensive applications it can be interesting to resort to a distributed storage to reach scalability and avoid data-intensive problems. Storing data permanently on computing nodes can be an interesting approach especially with the frequent use and the large volume of this data. Moreover, processing large data is a computing intensive task which encourages parallel execution. Nevertheless, data placement on computing nodes should be optimal to reach load balancing. In this work, we investigate scheduling heuristics towards the optimization of data distribution on the computing nodes. Motivated by its capacity to control perfectly the common operations associated with data management, we use BitDew: a desktop grid middleware designed for large scale data management. With BitDew, we build a Data-Driven Master/Worker Platform to carry out the distribution of Magick, the OCR application based on Dynamic Time Warping (DTW) algorithm. We evaluate the benefit of the implementation of studied scheduling heuristics to achieve load balancing with both homogeneous and heterogeneous environment. We present experimental results which demonstrate the efficiency of our approach