Revisiting network scanning detection using sequential hypothesis testing

Network scanning is a common, effective technique to search for vulnerable Internet hosts and to explore the topology and trust relationships between hosts in a target network. Given that the purpose of scanning is to search for responsive hosts and network services, behavior-based scanning detection techniques based on the state of inbound connection attempts remain effective against evasion. Many of today's network environments, however, feature a dynamic and transient nature with several network hosts and services added or stopped (either permanently or temporarily) over time. In this paper, working with recent network traces from two different environments, we re-examine the Threshold Random Walk (TRW) scan detection algorithm, and we show that the number of false positives is proportional to the transiency of the offered services. To address the limitations found, we present a modified algorithm (Stateful Threshold Random Walk (STRW) algorithm) that utilizes active mapping of network services to take into account benign causes of failed connection attempts. The STRW algorithm eliminates a significant portion of TRW false positives (e.g., 29% and 77% in two datasets studied)