Investigating the scaleability of analyzing and processing RDBMS datasets with Apache Spark

At the Uppsala Monitoring Centre (UMC), individual case safety reports (ICSRs) are managed, analyzed and processed for publishing statistics of adverse drug reactions. On top of the UMC’s ICSR database there is a data processing tool used to analyze the data. Unfortunately, there are some constraints limiting the current processing-tool along with that the amount of arriving data to be processed grows at a rapid rate. The UMC’s processing system must be improved in order to handle future demands. In order to improve performance various frameworks forparallelization can be used. In this work, the in-memory computing framework Sparkwas used for parallelization of one of the current data processing tasks. Local clusters for running the new implementation in parallel was also established