Building Distributed File Systems on Commercial Cloud Storage Services

With the increase in bandwidths available for internet users, cloud storage services have emerged to offer home users an easy way to share files and extend the storage space available for them. Most systems offer a limited free storage quota and combining these resources from multiple providers could be intriguing to cost-oriented users. In this study, we will implement a virtual file system that utilizes multiple different commercial cloud storage services (Dropbox, Google Drive, Microsoft OneDrive) to store its data. The data will be distributed among the different services and the structure of the data will be managed locally by the file system. The file system will be run in user space using FUSE and will use APIs provided by the cloud storage services to access the data. Our goal is to show that it is feasible to combine the free space offered by multiple services into a single easily accessible storage medium. Building such a system requires making design choices in multiple problem areas ranging from data distribution and performance to data integrity and data security. We will show how our file system is designed to address these requirements and will then conduct several tests to measure and analyze the level of performance provided by our system in different file system operation scenarios. The results will also be compared to the performance of using the distinct cloud storage services directly without distributing the data. This will help us to estimate the overhead or possible gain in performance caused by the distribution of data. It will also help us to locate the bottlenecks of the system. Finally, we will discuss some of the ways that could be used to improve the system based on test results and examples from existing distributed file systems