LSM-based Secure System Monitoring Using Kernel Protection Schemes

Abstract—Monitoring a process and its file I/O behaviors is important for security inspection for a data center server against intrusions, malware infection and information leakage. In the case of the Linux kernel 2.6, a set of hook functions called the Linux Security Module (LSM) has been implemented in order to monitor and control the system calls. By using the LSM we can inspect the activity of unknown malicious processes. However, a sophisticated attacker could breach the kernel configurations using the rootkits. Furthermore since the monitoring results of the malicious process activity are stored as a file on Hard Disk Drive (HDD), it will be easily manipulated by the attacker. In this paper, we propose a secure monitoring scheme that addresses the attacks against the monitoring module and its result for security inspection of the data center server. The monitoring module is implemented as a LSM-based function and protected by the kernel protection technique. The integrity of the monitoring result is guaranteed by using a Mandatory Access Control (MAC) of the Linux kernel and a mechanism of the trusted process invocation. This mechanism can serve as an infrastrucuture of secure inspection platform for data center server because the integrity of the monitoring module and its result is guaranteed. Keywords-Secure system monitoring; Linux Security Mod-ule; Lifetime kernel code integrity; Mandatory Access Control