Intrusion Detection Testing and Benchmarking Methodologies

©2003 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or distribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.Presented at the First IEEE International Workshop on Information Assurance (IWIAS), 2003The ad-hoc methodology that is prevalent in today’s testing and evaluation of network intrusion detection algorithms and systems makes it difficult to compare different algorithms and approaches. After conducting a survey of the literature on the methods and techniques being used, it can be seen that a new approach that incorporates an open source testing methodology and environment would benefit the information assurance community. After summarizing the literature and presenting several example test and evaluation environments that have been used in the past, we propose a new open source evaluation environment and methodology for use by researchers and developers of new intrusion detection and denial of service detection and prevention algorithms and methodologies