Advancing Mac OS X rootkit detection

AbstractIn the last few years there has been a sharp increase in the use of Mac OS X systems in professional settings. This has led to increased activity in the development of malware and attack toolkits focused specifically on OS X systems, and unfortunately, these increasingly powerful offensive capabilities have not (yet) resulted in better defensive research. Only a few public defensive research efforts currently exist and these only cover a portion of the attack surface that malicious OS X software has access to, particularly regarding kernel-level malware.In this paper, we present new rootkit detection techniques that attempt to close the gap between offense and defense, with a specific focus on kernel-mode components. The new detection techniques in this paper were motivated by analyzing currently available detection strategies for Windows and Linux, and noting associated deficiencies in detection schemes for Mac OS X. For each missing capability, OS X was studied to see if a similar operating system facility existed and if it could be abused by malware. For those fitting these criteria, new detection techniques were created, and these are discussed in detail in the paper.For each new rootkit detection technique we propose, a Volatility plugin was developed. Volatility is currently by far the most popular memory forensics framework in incident response and malware analysis, and by incorporating our work into Volatility, it can become immediately useful to the community. The paper concludes with an evaluation of the plugins, to illustrate their usefulness