Memory Utilization in R: The Impact of Data Management Frameworks (Packages)

Memory management is a very important component of running large workloads in computing. It takes into account the capacity & limitations of the available memory on the device itself and deallocates memory space when it is not needed or expands the space through virtual memory. Memory management strives to optimize memory usage so that the CPU can efficiently access the instructions and data it needs to execute the various processes. This work is focused on memory utilization by different data management frameworks in R on different platforms. We have considered Native R, tidyverse, and data.table as data management frameworks. Very high precision (1e-5+1 to 1e-6+1) visual analysis of memory utilization data shows Native R memory management is better when compared to tidyverse and data.table. But, when we are analysing data on a large scale and observe the memory utilization, it shows no significant difference in distribution of memory utilization across different sample sizes. We have established the same results using analysis of variance (ANOVA) and analysis of coefficients of linear regression models