Design Considerations for Developing a Disk File System

File Systems manage the most important entity of a digital computer system,namely digital-data. They are responsible for the storage and retrieval of digital data residing on the storage sub-system. However, this job of file systems is made difficult by the advancements and enhancements in hardware technology, and the dynamic and intense user requirements. The current hardware technology trends and workloads challenge the performance, scalability and extensibility of file systems. As such, file systems need to evolve to cope up with this requirement. Though file systems have been evolving since their existence, but the way they have been, has unknowingly led to other problems. These include objective specific disk file systems, incompatible variants of same basic design, reluctance on part of a file system to adapt to new features, and so on. Furthermore, the benchmarking methodology followed to evaluate the performance of a file systemis problematic. The motivation of this research is to enhance file systems to overcome and mitigate the challenges put forth by current hardware technology trends and user requirements such that the effort can be applied to all file systems or atleast a class of file systems in order to avoid the problems inherent in existing evolution. The central theme in this thesis is to overcome the challenges of performance, scalability and extensibility of file systems without modifying the design or source of the file system. In this thesis, we investigate various existing approaches and new concepts to achieve this goal. These include exploiting the concept of virtual unification to break file size limitation, the hybrid storage for performance and the block level behavior for reliable and efficient secure data deletion. Furthermore, this thesis also investigates a new benchmarking methodology for file system benchmarks. To validate the concepts, we have designed 4 file systems namely suvFS, hFAT,restFS and OneSec, and have implemented three of them while simulated therest. In addition, we have empirically evaluated the claims made by these filesystems. Our evaluations show that suvFS can break file size limitation of any file systemwithout modifying the design or source of that file system. Although,suvFS adds performance overhead, but the overhead is mainly due to FUSE framework. Furthermore, the normalized statistics revealed that the average performance overhead added by suvFS algorithms during reading is as low as 0.59 % while during writing and deleting it is as high as 11.22 % and 14.77 % respectively. However, the average growth of performance overhead with doubling the file size is 0.51 % for reading, 6.73 % for writing and 9.54 % for deletion. Furthermore, the simulation of hFAT file system indicates that it can reduce the latency incurred by FAT32 file system operations by a minimum of 25 %, 10 % and 90 % during writing, reading and deleting large number of small files respectively, if a solid state storage device having latency lesser or equal to 10% ofthat of magnetic disk is used in addition. Furthermore, this performance enhancement doesn’t demand any design or source modification of the file system. Similarly, the evaluation of restFS showed that it can save a minimum of 28 %of data blocks that needed to be overwritten and can reduce the number of diskwrites issued by 88 % as compared to other existing overwriting techniques when mounted on top ofext2 file system. Furthermore, this feature doesn’t demand any design or source modification of the file system. Finally, OneSec pointed out that the Linux OS file system framework adds large costs to filesystem operations which is as minimum as 76.93 % in OneSec. Furthermore, on evaluating ext2 file system against OneSec, the evaluation pointed out that reading operation of ext2 file system is 96.86 % nearer to efficient design while deletion is 95.62 % andwriting is95.39 % nearer. Also, the comparison of normalized results showed that ext2 file system should reduce its disk I/Os inorder to yield better performance