PySpark Algorithms

This is an introductory book on PySpark. This book is about PySpark: Python API for Spark.Apache Spark is an analytics engine for large-scaledata processing. Spark is the open source clustercomputing system that makes data analytics fastto write and fast to run. This book provides alarge set of recipes for implementing big dataprocessing and analytics using Spark and Python.The goal of this book is to show working examplesin PySpark so that you can do your ETL andanalytics easier. You may cut and paste examples todeliver your applications in PySpark. This book introduces PySpark (Python API for Spark).You can use PySpark to tackle big datasets quicklythrough simple APIs in Python. You will learn how toexpress parallel tasks and computations with just afew lines of code, and cover applications from ETL,simple batch jobs to stream processing and machinelearning. With this book, you may dive into Spark capabilitiessuch as RDDs (resilient distributed datasets),DataFrames (data as a table of rows and columns),in memory caching, and the interactive PySparkshell, where you may leverage Spark\u27s powerful builtin libraries, including Spark SQL, Spark Streaming,and MLlib. In this book, you will learn Spark\u27s transformationsand actions by a set of well-defined and workingexamples. All examples are tested and working: thismeans that you can copy-cut-paste to your desiredPySpark applications. Writing PySpark is much easierthan writing Spark applications in Java and PySparkapplications are not bulky at all when compared toJava Spark.https://scholarcommons.scu.edu/faculty_books/1457/thumbnail.jp