Comparison of Join Algorithms in Map Reduce Framework

ABSTRACT: In the current technological world, there is generation of enormous data each and every day by different media and social networks. The MapReduce framework is increasingly being used widely to analyse large volumes of data. One of the techniques that framework is join algorithm. Join algorithms can be divided into two groups: Reduceside join and Map-side join. The aim of our work is to compare existing join algorithms which are used by the MapReduce framework. We have compared Reducer-side merge join and Map-side replication-join in terms of preprocessing, the number of phases involved, whether it is sensitive to data skew, whether there is need for distributed Cache, memory overflow. I INTRODUCTION Large-scaled data warehouse systems, data-intensive analysis, cloud computing technologies have been used by Data-intensive applications. Large-scale data analysis applications use MapReduce paradigm [6] to process the data. MapReduce Framework is a programming model that is used to process and generate large data sets. User specifies a map function that processes a (key value) pair to generate a set of intermediate (key value) pairs and a reduce function which will merge all the intermediate values associated with the intermediate key MapReduce Execution: The Map/Reduce framework consists of two operations, "map" and "reduce", which are executed on a cluster of shared-nothing commodity nodes. In a map operation, the input data available through a distributed file system, is distributed among a number of nodes in the cluster in the form of key-value pairs. Each of these mapper nodes transforms a key-value pair into a list of intermediate key