Enclave Dynamic Memory Management (EDMM)
In SGX version 1, enclaves have a fixed size. One needs to allocate the maximum required memory at the start of an application. In many cases, this increases the startup times. One also needs to estimate the maximum memory needed by an application. If the estimates are too low, garbage-collected languages will run their garbage collector more often. Applications might even crash if there is too little memory available to complete their tasks.
Native applications do not allocate all memory at startup. Instead, a native application asks the operating system for more memory whenever it runs low on memory. In SGX version 2, confidential applications can do the same with the help of EDMM (Enclave Dynamic Memory Management).
Another advantage of EDMM is that MrEnclave, i.e., the secure hash value of the initial enclave state, is independent of the heap size: allocating more or less heap during runtime, will not change MrEnclave.
Modern Intel Icelake CPUs support EDMM, i.e., one can start with a small initial enclave size and then add more memory on demand during runtime. Starting with Linux kernel version 5.11, the kernel supports SGX out of the box. However, the current Linux kernel does not yet support EDMM. There is ongoing work to add EDMM support, but this is not yet stable.
Right now, one needs to use the out-of-tree SGX driver to support EDMM. That requires some extra installation work. Hence, we maintain a VM image on Azure that enables EDMM.
Azure: EDMM-enabled VM
Until the Linux kernel supports EDMM, we will maintain a VM image that supports EDMM on Azure (see details): You can use this image to run confidential applications that benefit from EDMM.
SCONE applications can run on SGX version 1 and version 2 CPUs. In particular, they can run with EDMM enabled and disabled.
The MrEnclave changes if one enables EDMM. Therefore, when determining MrEnclave, we need to specify that we want to determine MrEnclave either for EDMM enabled or disabled.
Consider that we want to determine MrEnclave of a confidential program
./test-c. To determine MrEnclave for disabled EDMM, we can execute:
export nonedmm_hash=$(SCONE_HASH=1 ./test-c)
nonedmm_hash will contain MrEnclave of
./test-c. To determine MrEnclave for EDMM enabled, you can execute:
export edmm_hash=$(SCONE_EDMM_HASH=1 ./test-c)
When starting an application, by default EDMM is disabled. One can enable EDMM by setting:
Note that the application will fail in case EDMM is not supported by the operating system or by the CPU.
One can explicitly disable EDMM by setting
SCONE also supports an automatic mode in which EDMM is used whenever it is supported by the CPU and the operating system:
Running an application in auto mode, one would list in the SCONE session policy for with and without EDMM:
services: - name: test-c attestation: - mrenclave: - $nonedmm_hash - $edmm_hash
When enabling EDMM, only a minimal heap is allocated at startup. By default, this is 20 MB. One can change the size of this initial heap with the help of environment variable
For example, we could increase the initial heap to 30MB as follows:
Note that it is typically a little faster to allocate initially at least all the heap pages that are needed for the program to start up.
When using EDMM, one should set SCONE_HEAP sufficiently high such that the application never terminates because it does not have sufficient memory.
When signing an application with
scone-signer, there are two signatures added:
- a signature for EDMM, and
- a signature without EDMM.
Using EDMM, one needs to specify
SCONE_MIN_HEAP might be set to the minimum heap required for the application to start up.
SCONE_HEAP would be set to the maximum heap that an application might require. It is better to overestimate
SCONE_HEAP to ensure that an application does not abort with an out-of-memory exception.