Nemesis: Judging the Efficacy of OS Fingerprinting Systems

The internet has gone from being a small network of niche uses and mostly academic interest to being a vital, foundational, piece of modern infrastructure that the world depends on. Because of its importance, the internet has also become a target and a gateway for malicious entities. Indeed, it has spawned a whole new military dimension: cyberwarefare. Of vital importance to both attackers and defenders is the identification of vulnerable systems connected to the internet. OS Fingerprinting is one mechanism by which vulnerable systems may be detected. Despite the importance and proliferation of OS Fingerprinting tools, there has not been a systematic effort to evaluate the effectiveness of such tools. We propose dimensionality as a metric for evaluating OS Fingerprinting Systems and provide a Framework to calculate this value. In addition, we identify NMAP as being the premier OS Fingerprinting tool used today and apply our Framework to this tool under various distortions to ascertain its performance based on our metric of dimensionality. Under its default conditions NMAP struggles with Firewalls, which are abundant on the internet, and performs poorly. Our Framework can identify confounding signatures within the database which disproportionately harm NMAPs dimensionality and can remove them from the database. Further, we find that NMAP has modest struggles with network jitter, potentially even on local networks. This, combined with NMAPs difficulty with Firewalls suggests that it is ill suited to the task of Fingerprinting Operating Systems over the internet. Lastly, we identify which features are crucial to NMAPs ability to identify Operating Systems. This, in addition to our other findings, potentially points in the directions for improvements to NMAP, both in it’s ability to identify an OS over the internet, and in reducing the number of probes needed to do so