IP video surveillance storage system

Multiple factors may cause a data storage error such as storage medium defects, dust particles and radiation. When a data in the storage become corrupted, it can no longer be use. This is an unacceptable situation for sensitive data such as surveillance video. Therefore, data redundancy is a critical factor in storing such data. Data redundancy refers to a condition where extra measures are taken to ensure data availability despite storage errors. There are various methods to achieve data redundancy. The simplest method is to replicate data across multiple storage locations. However, this method do not utilize storage space efficiently. Another method is by using erasure coding. Erasure coding is a method of data protection that splits data into smaller pieces and stored with additional redundancy pieces. The new form of erasure code implemented in this project combines Reed Solomon, a traditional erasure code, with XOR parity. This approach provides better storage efficiency compared to data replication technique. The usage of XOR parity will increase the chance of data error recoverability. Various functions of the erasure code will be elaborated in this report. The motivation of this project is to develop an erasure code storage system to replace replication storage system in OpenStack, a system that manages various resources to provide services such as cloud storage. In order to ensure that the erasure code is working correctly, multiple test cases are designed and carried out on the erasure code. Testing is simulated based on OpenStack’s storage environment. Testing of the erasure code was done not only in terms of functionality but also in terms of time. Testing was performed on two different machines to measure the effect of the code on higher and lower performance machines. A few limitations of the implemented erasure code will be elaborated, such as time performance due to Reed Solomon encoding and decoding. There will also be suggestion for future development to improve the erasure code performance.Bachelor of Engineering (Computer Engineering