Efficient Batched Synchronization in Dropbox-like Cloud Storage Services

Abstract. As tools for personal storage, file synchronization and data sharing, cloud storage services such as Dropbox have quickly gained popularity. These services provide users with ubiquitous, reliable data storage that can be automati-cally synced across multiple devices, and also shared among a group of users. To minimize the network overhead, cloud storage services employ binary diff, da-ta compression, and other mechanisms when transferring updates among users. However, despite these optimizations, we observe that in the presence of frequent, short updates to user data, the network traffic generated by cloud storage services often exhibits pathological inefficiencies. Through comprehensive measurements and detailed analysis, we demonstrate that many cloud storage applications gen-erate session maintenance traffic that far exceeds the useful update traffic. We refer to this behavior as the traffic overuse problem. To address this problem, we propose the update-batched delayed synchronization (UDS) mechanism. Acting as a middleware between the user’s file storage system and a cloud storage ap-plication, UDS batches updates from clients to significantly reduce the overhead caused by session maintenance traffic, while preserving the rapid file synchro-nization that users expect from cloud storage services. Furthermore, we extend UDS with a backwards compatible Linux kernel modification that further im-proves the performance of cloud storage applications by reducing the CPU usage