Development of a Cluster System with Dynamic Loading Balance for Video Streaming

網際網路質與量高倍數的需求成長已經使得伺服器需求及負擔愈趨繁重，將單一伺服器作為服務的方式早已經不能夠滿足成長需求。為了能夠有效率地提昇穩定的服務品質，大型網站早已使用了大量伺服器叢集系統來解決傳統單一伺服器效能擴充問題，但是隨之而來的問題是如何根據實際伺服器負載狀況來進行負載分配，使其能不間斷的提供服務，並且考量低成本高效能以及系統容錯功能，更是最近這幾年很熱門的研究課題。 在本篇論文裡，我們利用了Linux作業系統的NAT功能結合封包改寫技術與動態負載平衡演算法，完成了軟體核心模組的實作。此軟體模組的優點可以在IP層進行封包遞送，並且可以依據叢集系統後端伺服器實際負載狀況進行負載分配。最後我們也將此軟體模組與視訊串流軟體作了整合，使整個叢集系統具備視訊串流的服務，期望未來能夠成為功能更為完善的視訊隨選叢集系統。Abstract With the explosive growth of the World Wide Web, some large web sites experience a lot of request and heavy loads. A single server hosting a service is usually not sufficient to handle this aggressive growth. In order to provide quality of server effectively, many of the popular web sites deploy a set of machines, or a cluster, and have them work together to host a single service. However, there are many problems which follow cluster system. This include: how to dispatch the flow rate, consider both cost-effective and high performance, and supply of failure-handling. The above- mentioned all are very popular research in the recent years. In this paper, we use NAT functionality in the Linux operating system to combine with packet-rewriting technology and dynamic load-balancing algorithm, and finish development of software kernel module. This kernel module has many advantages: (1) packet can be forwarded at the IP level; (2) load-balancing mechanism can base on the real system load of the back server which in the cluster system. Finally, we also integrate the kernel module with video stream software, and make the whole cluster system to hold video-stream service, and desire that it become more complete目 錄 摘要……………………………………………………………….……i ABSTRACT……………………………………………………………ii 致謝……………………………………………………………….…..iii 目錄……………………………………………………………….…...iv 圖目錄…………………………………………………………….…..vii 表目錄…………………………………………………………….…...ix 第一章 緒論………………………………………………………..1 1.1研究背景……………………………………...…….....1 1.2 研究動機與目的……………………………………...2 1.3 研究範圍……………………………………………...4 1.4 章節結構……………………………………………...4 第二章 相關文獻探討……………………………………………..5 2.1 叢集系統定義與描述………………………………...5 2.1.1叢集系統的特性……………………….............5 2.2 叢集系統負載平衡架構……………………………...6 2.3 叢集系統之排程法之探討………………………......11 第三章 封包轉送技術之研究……………………………………17 3.1 Linux Virtual Server之架構…………………………18 3.1.1 以NAT轉送封包之原理與流程……………18 3.1.2 以IP Tunneling轉送封包原理與流程………20 3.1.3 以Direct Routing轉送封包原理與流程…….21 3.2 LVS封包轉送工作原理……………………………..23 3.3 LVS排程演算法…………………………………..…26 3.4 相關技術及產品設計……………………………….30 第四章 Linux作業系統上實作分析…………………………….33 4.1 網路核心程式分析………………………………….33 4.2 Netfilter專案………………………………………....36 4.3 系統架構………………………………………….....39 4.4 基本封包轉送機制設計…………………………….43 第五章 系統設計…………………………………………………45 5.1 環境建立與位址分配……………………………….45 5.2 封包遞送…………………………………………….46 5.3 動態排程機制…………………………………….…47 5.4 連線記錄管理………………………………….……51 5.5 備援機制………………………………….…………54 5.6 GUI遠端監控…………………………………..……55 5.7 視訊串流播放…………………………………….…56 第六章 結論與未來工作…………………………………………57 6.l 系統設計的特點……………………………………..57 6.2 負載平衡演算法之比較………………………….....58 6.3 未來工作………………………………………….....58 參考文獻 …………………………………………............................5