SCONE Environment Variables

To simplify development and debugging, SCONE supports a range of environment variables to control its behavior. These environment variables are mainly used for development and debugging. In operational settings, the configuration would be provided in a secure fashion via the SCONE configuration and attestation service.

Also, the performance of SCONE-based applications can be tuned by selecting the appropriate configuration for an application. We have tool support to automatically tune these parameters.

SCONE Configuration File

The location of the SCONE configuration file can be controlled with an environment variable:

• SCONE_CONFIG: if defined, this determines the path of SCONE configuration file. If this environment variable is not defined or the file cannot be opened, the default configuration file located in /etc/sgx-musl.conf is read instead. If the default configuration file cannot be read either, the program exits with an error message.

Changing the location of the configuration file is, for example, useful in the context of testing or when you run your application outside of a container when you want to run different applications with different configurations inside of enclaves.

The configuration file can define most of the behaviors that one can define via environment variables. However, the SCONE_... environment variables have higher priority than the settings from the config file.

Format of SCONE Configuration File

The format for the configuration file: on each line, there is a statement beginning with a single-character command code, and up to three numbers. The possible commands currently are:

Command	Option(s)	Description
Q	n	defines the number of queues used to execute system calls
	n [number of queues]	sets the number of syscall-return queue pairs to allocate. This is equivalent to setting the SCONE_QUEUES environment variable
H	s	defines the heap size in bytes
	s [heap size in bytes]	sets the size of heap, equivalent to setting SCONE_HEAP environment variable
P	N	determines the backoff behavior of the queues
	N [spin number]	equivalent to setting SCONE_SSPINS environment variable
L	S	determines the backoff behavior of the queues
	S [sleep time]	equivalent to setting SCONE_SSLEEP environment variable;
s	C Q R	sthread serving system calls outside of an enclave
	C [core-no]	if non-negative number: pin this sthread to core C; if a negative number, do not pin this thread
	Q [queue-no]	in [0..n] ; this sthreads serves this queue
	R [realtime]	always set this to 0
e	C Q R	ethread running inside of enclave and executes application threads (which we call lthreads)
	C [core-no]	non-negative number: pin to this core; negative number: no pinning ot this thread
	Q [queue-no]	in [0..n]: this sthreads serves this queue
	R [realtime]	always set this to 0

The number of sthreads is automatically increased if more sthreads are needed to process system calls. An sthread will block if it does not have any work left to do. Hence, we recommend to start exactly one sthread per ethread.

ethreads will leave the enclave it there is no more work for them to do. Hence, it makes sense to start one ehthread per core. In some situations, it might even make sense to start one ethread per hyper-thread.

Unless you want to limit the the number of CPU resources an enclave should use, the following is a good generic configuration file:

$ sudo tee  /etc/sgx-musl.conf << EOF
Q 4
e -1 0 0
s -1 0 0
e -1 1 0
s -1 1 0
e -1 2 0
s -1 2 0
e -1 3 0
s -1 3 0
EOF

Run Mode

• SCONE_MODE: defines if application should run inside of an enclave or outside in simulation mode.

Value	Description
SCONE_MODE=HW	run program inside of enclave. If program fails to create an enclave, it will fail.
SCONE_MODE=SIM	run program outside of enclave. All SCONE related software runs but just outside of the enclave. This is not 100% compatible with hardware mode since, for example, some instructions are permitted in native mode that are not permitted in hardware mode.
SCONE_MODE=AUTO	run program inside of enclave if SGX is available. Otherwise, run in simulation mode. AUTO is the default mode

Memory-Related Environment Variables

• SCONE_HEAP: size of the heap allocated for the program during the startup of the enclave. The default heap size is 64MB and is used if SCONE_HEAP is not set.

Value	Description
SCONE_HEAP=s	the requested heap size in bytes
SCONE_HEAP=sK	the requested heap size in KiloBytes
SCONE_HEAP=sM	the requested heap size in MegaBytes
SCONE_HEAP=sG	the requested heap size in GigaBytes

• SCONE_STACK: size of the stack allocated to threads spawned in the enclave.

The default stack size is 64 KBytes. Please increase this if your expect your threads to require more than 64KBytes. For programs like MariaDB, MongoDB, Python, or node, we increase the stack size to 4 MBytes.

Value	Description
SCONE_STACK=s	the requested stack size in bytes
SCONE_STACK=sK	the requested stack size in KiloBytes
SCONE_STACK=sM	the requested stack size in MegaBytes

Debug

• SCONE_VERSION if defined, SCONE will print the values of some of the SCONE environment variables during startup.

• SCONE_LOG set to value among FATAL (less verbose), ERROR, WARNING, INFO, DEBUG and TRACE (more verbose) to see more or less messages on stderr from the SCONE platform. Messages could be warnings if certain functions are not (yet) implemented inside of an enclave. Please note that setting the log level to TRACE might increase resource consumption.

Example output for SCONE_VERSION=1:

export SCONE_QUEUES=4
export SCONE_SLOTS=256
export SCONE_SIGPIPE=0
export SCONE_MMAP32BIT=0
export SCONE_SSPINS=100
export SCONE_SSLEEP=4000
export SCONE_KERNEL=0
export SCONE_HEAP=2147483648
export SCONE_STACK=81920
export SCONE_LOG=WARNING
export SCONE_CONFIG=/etc/sgx-musl.conf
export SCONE_ESPINS=10000
export SCONE_MODE=hw
export SCONE_SGXBOUNDS=no
export SCONE_VARYS=no
export SCONE_ALLOW_DLOPEN=yes (unprotected)
export SCONE_MPROTECT=yes
Revision: e37dda73a6db435973f2e00347bd4cf462e4e027 (Sat Aug 25 22:59:30 2018 +0200)
Branch: master
Configure options: --enable-file-prot --enable-shared --enable-debug --prefix=/scone/src/built/cross-compiler/x86_64-linux-musl

Dynamic library support:

• SCONE_ALLOW_DLOPEN="1": if defined, permit to load protected libraries after startup: These libraries must be authenticated or encrypted to be able to load these. Note that all libraries that are loaded during startup are measured and contribute to the hash of the enclave, i.e., they are part of MRENCLAVE. The libraries loaded during startup could reside in a file region that is not protect or that is authenticated. These libraries must not be in an encrypted region since the encryption keys are not yet known during startup.

Note that SCONE_ALLOW_DLOPEN="1" must be set in the policy of an attested application to have an effect.

• SCONE_ALLOW_DLOPEN="2": must be used for debugging only: this value enables loading of unprotected dynamic libraries after startup, i.e., libraries that are neither authenticated nor encrypted. For example, Python programs might dynamically load modules after startup. Our approach to enforce the integrity of these dynamic libraries with the help of a file protection shield, i.e., you should either set SCONE_ALLOW_DLOPEN="1" or you should not define SCONE_ALLOW_DLOPEN. Never use SCONE_ALLOW_DLOPEN=2 in production mode.

Performance tuning variables

• SCONE_QUEUES: number of systemcall queues to be used.

• SCONE_SLOTS: systemcalls queue length: must be larger than the maximum number of lthreads.

• SCONE_SIGPIPE: if set to 1, SIGPIPE signals are delivered to the application

• SCONE_SSPINS=N: In case an Ethread does not have any lthread to execute, an Ethread first pauses for up to N times to wait for more work to show up. After that, it sleeps for up to N times. Each time increasing the sleep time.

• SCONE_SSLEEP: determines how fast we increase the backoff.

Safety

• SCONE_SGXBOUNDS: must be defined to enable bounds checking. Furthermore, you need to compile your program with our SGX boundschecker.

Dynamic link loader

The dynamic link loader is part of image registry.scontain.com/sconecuratedimages/crosscompilers:runtime (see tutorial).

• SCONE_LD_DEBUG: print the dynamically loaded libraries and their SHA-256 hashes

• LD_LIBRARY_PATH: you can control from where the dynamic link loader looks for shared libraries.

• LD_PRELOAD: you can instruct the dynamic link loader to load libraries before loading the libraries specified by the program itself.

• SCONE_ALPINE=1: run dynamically-linked program inside of an enclave.

Fork Support

We support fork since release 4.2. It is, however, by default disabled: when a process executes fork, the call will fail by default. Ony if the environment variable SCONE_FORK is set to 1, fork will be executed.