Survey on Big Data: Management and Challenges

Abstract-Today we live in the digital world. With increased digitization the amount of structured and unstructured data being created and stored is exploding. The data is being generated from various sources -transactions, social media, sensors, digital images, videos, audios and click streams for domains including healthcare, retail, energy and utilities. Big Data is therefore an all-encompassing term for any collection of large data sets that were difficult to process. However, the fast growth rate of such large data generates numerous challenges, such as the rapid growth of data, transfer speed, diverse data, and security. The current volume of big data sets is too complicated to be managed and processed by conventional relational databases & data warehousing technologies. This development calls for new system architectures for data acquisition, transmission, storage, and large-scale data processing mechanisms. This paper presents various big data definitions, challenges, approaches and the system architecture