Cybersecurity Games: Mathematical Approaches for Cyber Attack and Defense Modeling

Cyber-attacks targeting individuals and enterprises have become a predominant part of the computer/information age. Such attacks are becoming more sophisticated and prevalent on a day-to-day basis. The exponential growth of cyber plays and cyber players necessitate the inauguration of new methods and research for better understanding the cyber kill chain, particularly with the rise of advanced and novel malware and the extraordinary growth in the population of Internet residents, especially connected Internet of Things (IoT) devices. Mathematical modeling could be used to represent real-world cyber-attack situations. Such models play a beneficial role when it comes to the secure design and evaluation of systems/infrastructures by providing a better understanding of the threat itself and the attacker\u27s conduct during the lifetime of a cyber attack. Therefore, the main goal of this dissertation is to construct a proper theoretical framework to be able to model and thus evaluate the defensive strategies/technologies\u27 effectiveness from a security standpoint. To this end, we first present a Markov-based general framework to model the interactions between the two famous players of (network) security games, i.e., a system defender and an attacker taking actions to reach its attack objective(s) in the game. We mainly focus on the most significant and tangible aspects of sophisticated cyber attacks: (1) the amount of time it takes for the adversary to accomplish its mission and (2) the success probabilities of fulfilling the attack objective(s) by translating attacker-defender interactions into well-defined games and providing rigorous cryptographic security guarantees for a system given both players\u27 tactics and strategies. We study various attack-defense scenarios, including Moving Target Defense (MTD) strategies, multi-stage attacks, and Advanced Persistent Threats (APT). We provide general theorems about how the probability of a successful adversary defeating a defender’s strategy is related to the amount of time (or any measure of cost) spent by the adversary in such scenarios. We also introduce the notion of learning in cybersecurity games and describe a general game of consequences meaning that each player\u27s chances of making a progressive move in the game depend on its previous actions. Finally, we walk through a malware propagation and botnet construction game in which we investigate the importance of defense systems\u27 learning rates to fight against the self-propagating class of malware such as worms and bots. We introduce a new propagation modeling and containment strategy called the learning-based model and study the containment criterion for the propagation of the malware based on theoretical and simulation analysis