Performance-Aware Data Placement Policy for Hadoop Distributed File System

Apache Hadoop is an open-source softwareframework for distributed storage and distributedprocessing of very large data sets on computerclusters built from commodity hardware. The HadoopDistributed File System (HDFS) is the underlying filesystem of a Hadoop cluster. The default HDFS dataplacement strategy works well in homogeneouscluster. But it performs poorly in heterogeneousclusters because of the heterogeneity of the nodescapabilities. It may cause overload in somecomputing nodes and reduce Hadoop performance.Therefore, Hadoop Distributed File System (HDFS)has to rely on load balancing utility to balance datadistribution. As a result, data can be placed evenlyacross the Hadoop cluster. But it may cause theoverhead of transferring unprocessed data from slownodes to fast nodes because each node has differentcomputing capacity in heterogeneous Hadoopcluster. In order to solve these problems, adata/replica placement policy based on storageutilization and computing capacity of each data nodein heterogeneous Hadoop Cluster is proposed. Theproposed policy tends to reduce the overload of somecomputing nodes as well as reduce overhead of datatransmission between different computing nodes