Specification and evaluation of polymorphic shellcode properties using a new temporal logic

International audienceIt is a well-known fact that polymorphism is one of the greatest find of malicious code authors. Applied in the context of Buffer Overflow attacks, the detection of such codes becomes very difficult. In view of this problematic, which constitutes a real challenge for all the international community, we propose in this paper a new formal language (based on temporal logics such as CTL) allowing to specify polymorphic codes, to detect them and to better understand their nature. The efficiency and the expressiveness of this language are shown via the specification of a variety of properties characterizing polymorphic shellcodes. Finally, to make the verification process automatic, this language is supported by a new IDS (Intrusion Detection System) that will also be presented in this paper