Elevated Penetration Attack Models of Virtual Machine Escape Based on FSM

Abstract

Virtual machine escape is one of the most serious vulnerabilities happening if the isolation between the hosts and between the VMs is compromised, which presents new security challenges that the security concern is the major factor effecting virtualization technology widely adopted in IT industry. In VM escape, the program running in a virtual machine is able to completely bypass the hypervisor layer, and get access to the host machine. The traditional research method is analyzing a vulnerability separately, but that consumes too much time and not constructs the attack model. So we innovatively design VM escape elevated penetration attack models based on finite state machine, which could be used to identify potential vulnerabilities in design, implementation and testing phases. In this paper, firstly, we extract elevated privilege models of different virtualization methods, studying that VMCS pointer instruction state indicates system state. Secondly, we define a formal language Datalog to represent pre- and post-conditions of the exploits of application vulnerabilities and infer a basicelevated penetration attack model. Thirdly, through the analysis of vulnerable source code and vulnerability reports from NVD, we shed light on four attack models to cover the most VM escape attacks. Finally, we evaluate the presented approach by applying code-level finite state machine models with formal language to specific vulnerabilities, together with the statistical results of different attack models.