SGXJail: Defeating Enclave Malware via Confinement

Trusted execution environments, such as Intel SGX, allow executing enclaves shielded from the rest of the system. This fosters new application scenarios not only in cloud settings but also for securing various types of end-user applications. However, with these technologies new threats emerged. Due to the strong isolation guarantees of SGX, enclaves can effectively hide malicious payload from antivirus software. Were these scenarios already outlined years ago, we are evidencing functional attacks in the recent past. Unfortunately, no reasonable defense against enclave malware has been proposed.

In this work, we present the first practical defense mechanism protecting against various types of enclave misbehavior. By studying known and future attack vectors we identified the root cause for the enclave malware threat as a too permissive host interface for SGX enclaves, leading to a dangerous asymmetry between enclaves and applications. To overcome this asymmetry, we design SGXJail, an enclave compartmentalization mechanism making use of flexible memory access policies. SGXJail effectively defeats a wide range of enclave malware threats while at the same time being compatible with existing enclave infrastructure. Our proof-of-concept software implementation confirms the efficiency of SGXJail on commodity systems. We furthermore present slight extensions to the SGX specification, which allow for even more efficient enclave compartmentalization by leveraging Intel memory protection keys. Apart from defeating enclave malware, SGXJail enables new use cases beyond the original SGX threat model. We envision SGXJail not only for site isolation in modern browsers, i.e., confining different browser tabs but also for third-party plugin or library management.