Easy and Effective Parallel Programmable ETL

Extract–Transform–Load (ETL) programs are used to load datainto data warehouses (DWs). An ETL program must extract datafrom sources, apply different transformations to it, and use the DWto look up/insert the data. It is both time consuming to develop andto run an ETL program. It is, however, typically the case that theETL program can exploit both task parallelism and data parallelismto run faster. This, on the other hand, makes the development timelonger as it is complex to create a parallel ETL program. To remedythis situation, we propose efficient ways to parallelize typical ETLtasks and we implement these new constructs in an ETL framework.The constructs are easy to apply and do only require fewmodifications to an ETL program to parallelize it. They supportboth task and data parallelism and give the programmer differentpossibilities to choose from. An experimental evaluation showsthat by using a little more CPU time, the (wall-clock) time to runan ETL program can be greatly reduced