Creating Data Pipelines using Apache Airflow

This Paper addresses the use of Apache Airflow in creating Data Pipelines, the paper gives an overview of what Apache Airflow is, basic building blocks like DAGs and Operators, explains how to create a simple pipeline using a realistic ETL use-case. Paper also briefly explains about Cloud Composer the fully managed service developed by Google Cloud Platform for Apache Airflow. Results from this study on Airflow suggests that using Apache Airflow can simplifies the Data Pipeline creation process as only pre-requisite to start using Airflow is the basic Python knowledge because the Operators in Airflow should be written in Python 3.6 or above