SCONE Environment Variables
To simplify development and debugging, SCONE supports a range of environment variables to control its behavior. These environment variables are mainly used for development and debugging. In operational settings, the configuration would be provided in a secure fashion via the SCONE configuration and attestation service.
Also, the performance of SCONE-based applications can be tuned by selecting the appropriate configuration for an application. We have tool support to automatically tune these parameters.
SCONE Configuration File
The location of the SCONE configuration file can be controlled with an environment variable:
SCONE_CONFIG: if defined, this determines the path of SCONE configuration file. If this environment variable is not defined or the file cannot be opened, the default configuration file located in /etc/sgx-musl.conf is read instead. If the default configuration file cannot be read either, the program exits with an error message.
Changing the location of the configuration file is, for example, useful in the context of testing or when you run your application outside of a container when you want to run different applications with different configurations inside of enclaves.
The configuration file can define most of the behaviors that one can define via environment variables.
SCONE_... environment variables have higher priority than the settings from the config file.
Format of SCONE Configuration File
The format for the configuration file: on each line, there is a statement beginning with a single-character command code, and up to three numbers. The possible commands currently are:
|Q||n||defines the number of queues used to execute system calls|
|n [number of queues]||sets the number of syscall-return queue pairs to allocate. This is equivalent to setting the
|H||s||defines the heap size in bytes|
|s [heap size in bytes]||sets the size of heap, equivalent to setting
|P||N||determines the backoff behavior of the queues|
|N [spin number]||equivalent to setting
|L||S||determines the backoff behavior of the queues|
|S [sleep time]||equivalent to setting
|s||C Q R||sthread serving system calls outside of an enclave|
|C [core-no]||if non-negative number: pin this sthread to core
|Q [queue-no]||in [0..n] ; this sthreads serves this queue|
|R [realtime]||always set this to 0|
|e||C Q R||ethread running inside of enclave and executes application threads (which we call lthreads)|
|C [core-no]||non-negative number: pin to this core; negative number: no pinning ot this thread|
|Q [queue-no]||in [0..n]: this sthreads serves this queue|
|R [realtime]||always set this to 0|
The number of sthreads is automatically increased if more sthreads are needed to process system calls. An sthread will block if it does not have any work left to do. Hence, we recommend to start exactly one sthread per ethread.
ethreads will leave the enclave it there is no more work for them to do. Hence, it makes sense to start one ehthread per core. In some situations, it might even make sense to start one ethread per hyper-thread.
Unless you want to limit the the number of CPU resources an enclave should use, the following is a good generic configuration file:
$ sudo tee /etc/sgx-musl.conf << EOF Q 4 e -1 0 0 s -1 0 0 e -1 1 0 s -1 1 0 e -1 2 0 s -1 2 0 e -1 3 0 s -1 3 0 EOF
SCONE_MODE: defines if application should run inside of an enclave or outside in simulation mode.
||run program inside of enclave. If program fails to create an enclave, it will fail.|
||run program outside of enclave. All SCONE related software runs but just outside of the enclave. This is not 100% compatible with hardware mode since, for example, some instructions are permitted in native mode that are not permitted in hardware mode.|
||run program inside of enclave if SGX is available. Otherwise, run in simulation mode. AUTO is the default mode|
Memory-Related Environment Variables
SCONE_HEAP: size of the heap allocated for the program during the startup of the enclave. The default heap size is 64MB and is used if SCONE_HEAP is not set.
||the requested heap size in bytes|
||the requested heap size in KiloBytes|
||the requested heap size in MegaBytes|
||the requested heap size in GigaBytes|
SCONE_STACK: size of the stack allocated to threads spawned in the enclave.
The default stack size is 64 KBytes. Please increase this if your expect your threads to require more than 64KBytes. For programs like MariaDB, MongoDB, Python, or node, we increase the stack size to 4 MBytes.
||the requested stack size in bytes|
||the requested stack size in KiloBytes|
||the requested stack size in MegaBytes|
SCONE_VERSIONif defined, SCONE will print the values of some of the SCONE environment variables during startup.
SCONE_LOGset to value between 0 (no) and 7 (debug) to see more or less messages on stderr from the SCONE platform. Messages could be warnings if certain functions are not (yet) implemented inside of an enclave.
Example output for
export SCONE_QUEUES=4 export SCONE_SLOTS=256 export SCONE_SIGPIPE=0 export SCONE_MMAP32BIT=0 export SCONE_SSPINS=100 export SCONE_SSLEEP=4000 export SCONE_KERNEL=0 export SCONE_HEAP=2147483648 export SCONE_STACK=81920 export SCONE_LOG=6 export SCONE_CONFIG=/etc/sgx-musl.conf export SCONE_ESPINS=10000 export SCONE_MODE=hw export SCONE_SGXBOUNDS=no export SCONE_VARYS=no export SCONE_ALLOW_DLOPEN=yes (unprotected) export SCONE_MPROTECT=yes Revision: e37dda73a6db435973f2e00347bd4cf462e4e027 (Sat Aug 25 22:59:30 2018 +0200) Branch: master Configure options: --enable-file-prot --enable-shared --enable-debug --prefix=/scone/src/built/cross-compiler/x86_64-linux-musl
Dynamic library support:
SCONE_ALLOW_DLOPEN="1": if defined, permit to load protected libraries after startup: These libraries must be authenticated or encrypted to be able to load these. Note that all libraries that are loaded during startup are measured and contribute to the hash of the enclave, i.e., they are part of MRENCLAVE. The libraries loaded during startup could reside in a file region that is not protect or that is authenticated. These libraries must not be in an encrypted region since the encryption keys are not yet known during startup.
SCONE_ALLOW_DLOPEN="1" must be set in the policy of an attested application to have an effect.
SCONE_ALLOW_DLOPEN="2": must be used for debugging only: this value enables loading of unprotected dynamic libraries after startup, i.e., libraries that are neither authenticated nor encrypted. For example, Python programs might dynamically load modules after startup. Our approach to enforce the integrity of these dynamic libraries with the help of a file protection shield, i.e., you should either set
SCONE_ALLOW_DLOPEN="1"or you should not define
SCONE_ALLOW_DLOPEN. Never use
SCONE_ALLOW_DLOPEN=2in production mode.
Performance tuning variables
SCONE_QUEUES: number of systemcall queues to be used.
SCONE_SLOTS: systemcalls queue length: must be larger than the maximum number of lthreads.
SCONE_SIGPIPE: if set to
1, SIGPIPE signals are delivered to the application
SCONE_SSPINS=N: In case an Ethread does not have any lthread to execute, an Ethread first pauses for up to N times to wait for more work to show up. After that, it sleeps for up to N times. Each time increasing the sleep time.
SCONE_SSLEEP: determines how fast we increase the backoff.
SCONE_SGXBOUNDS: must be defined to enable bounds checking. Furthermore, you need to compile your program with our SGX boundschecker.
Dynamic link loader
The dynamic link loader is part of image registry.scontain.com:5050/sconecuratedimages/crosscompilers:runtime (see tutorial).
SCONE_LD_DEBUG: print the dynamically loaded libraries and their SHA-256 hashes
LD_LIBRARY_PATH: you can control from where the dynamic link loader looks for shared libraries.
LD_PRELOAD: you can instruct the dynamic link loader to load libraries before loading the libraries specified by the program itself.
SCONE_ALPINE=1: run dynamically-linked program inside of an enclave.
fork since release 4.2. It is, however, by default disabled: when a process executes
fork, the call will fail by default. Ony if the environment variable
SCONE_FORK is set to 1,
fork will be executed.