PhishPrint: A Novel Framework for Scalable Evaluation of Web Security Crawlers and Mining of Unknown Cloaking Vectors

We built a novel scalable, low-cost, and generic platform named PhishPrint to enable the evaluation of Web Security Crawlers (WSCs) against previously unknown cloaking weaknesses. PhishPrint completely avoids the use of any simulated phishing sites and blocklisting measurements. We used PhishPrint to evaluate an unprecedented number of WSCs (23) including highly ubiquitous services such as Google Safe Browsing and Microsoft Outlook e-mail scanners. Our 70-day study found several unknown cloaking weaknesses with which we constructed 5 effective cloaking attack vectors (including 4 novel ones). In particular, it was shown that the entire WSC ecosystem is extremely vulnerable to a novel browser fingerprinting-based cloaking attack. We confirmed the practical impact of our findings by deploying 20 evasive phishing web pages that embedded the 5 cloaking vectors. 18 of the pages managed to survive indefinitely despite aggressive self-reporting of the pages to all WSCs. We confirmed the specificity of these attack vectors with 1150 volunteers as well as 400K web users. We also discuss countermeasures that all WSCs should take up in terms of both their crawler infrastructure as well as reporting infrastructure. We have relayed the found cloaking weaknesses to the 23 WSCs through an elaborate vulnerability disclosure process that resulted in some remedial actions as well as multiple vulnerability rewards