VMI based API redirection for run time protection

雲端產業的蓬勃發展使得企業傾向於使用此項科技來輔助企業的發展，伺服器整合已是無法抵禦的潮流。隨著伺服器的集中使用，集中點成為明顯的攻擊目標。了解程式行為會是一個很好的開始，透過了解程式行為方能過濾何者為高風險之行動，針對其作高精確度的防禦。本論文之最終目標為實作一套具彈性的實時程式行為側錄系統，其結果可基於安全的需要作為行為分析之用。 為了反制當代惡意程式所發展的反偵測技術，本論文透過使用虛擬化技術來使此監控行為不容易被發覺。如同許多過去的研究一般，屬於Virtual Machine Introspection (VMI) 技術之一類。隨著近代虛擬化技術的發展，硬體輔助虛擬化技術已成為標準配備之一。硬體輔助虛擬化技術帶來了顯著的效能增益，但改變了整體執行的架構與流程。 因此，本論文致力於在硬體輔助虛擬化機制下製造可有效的執行監控程式之空間，來協助程式行為監控的進行。為了要讓理解程式行為更簡單本論文將監測的基本單位至於較接近程式開發者熟悉的API call等級。API call等級的程式執行行為描繪比起傳統的系統呼叫（system call）具更高的語意價值，可更精確的了解程式行為，降低推論的複雜度。 由於API call的種類繁多並且功能豐富，本論文之系統可支援使用者自行設計之監控程式來因應API call的多樣性。並且提供API call執行前、後兩個執行時間點來幫助使用者取得API call之傳入參數以及其回傳值。此舉可使所錄製之行為描繪更貼近事實並且易於分析。 除了基本功能之外，本論文於系統設計時便將三個重點列入考慮：Transparency、Performance、Bridging the semantic gap。Transparency的兩個面向使得本系統不會受制於監控對象之作業系統選擇、不需安裝任何額外軟體，亦不會輕易的被作業系統內的防護機制查覺。執行本論文之系統不會使效能大幅度的下降，可享受硬體輔助虛擬化機制帶來的效能增益。最後，選擇API call作為觀察對象可使未來分析者更簡易但清楚地了解程式行為。As cloud computing prospers, server consolidation becomes a trending topic. In the mean time, it brought attention of attackers. In order to provide protection to cloud technology users, we must understand how these malicious activities function. This thesis presents a way to record program behavior through API redirection. While traditional IDS can provide protection against sophisticated attacks, it is also vulnerable to anti-detection mechanisms like anti-debugging and anti-instrumentation developed by attackers. Virtual machine introspection (VMI) technology moves IDS out of operating system to avoid such anti-detection mechanisms. With the aid of hardware-assisted virtualization technology, virtualization’s performance has increased significantly. However, the adaptation of such technology brings significant change to how virtualization functions. The change affected many existing VMI-based system, making it impossible to work as it was designed. This thesis aimed to solve this and build a VMI-based API redirection system on 64-bit hardware-assisted technology enabled machine. Additionally, three more aspects are considered throughout the design: Transparency, Performance, and bridging the semantic gap. By achieving all goals, we will have a system that requires no additional software installation, incur low performance overhead, and generates execution traces with higher semantic value. The results can be further analyzed to understand program behavior