Skip to content

SCONE Binary File System

Info

This document reflects the state of SCONE v5.7.0

The SCONE Binary File System (Binary FS) is a mechanism with which the user can embed a file system image into an enclave. Thereby, SGX' enclave protection mechanism is extended to the enclave's file system since the enclave's measurement (MRENCLAVE) will reflect the file system state. This is a particularly appealing solution for interpreted languages such as Python or Java where the actual program code (e.g. PyTorch or Kafka) is not part of the interpreter binary but read from byte code files in the file system. The file system contents are stored in a form of a shared library. Using the environment variable SCONE_EXTENSIONS_PATH this library can be added into the enclave during enclave creation. One must specify the path to the library in the variable, separating multiple libraries with : (e.g. lib1.so:lib2.so). Note that, 1) modifications the program does to the binary fs are not persistent, after each start the binary fs is in its initial state, and 2) the binary fs replaces the entire file system, i.e. an enclave with binary fs won't be able to read or write from the host's file system. You can utilize SCONE volumes to persistently store data on the host's file system.

TL;DR

A fully functioning single Dockerfile which runs a Python hello_world.py program using the binary-fs could look like this:

# First stage: apply the binary-fs
FROM registry.scontain.com/sconecuratedimages/apps:python-3.7.3-alpine3.10 AS binary-fs

RUN echo "print('Hello World!')" > /hello-world.py && \
    mkdir /binary-fs

# & apply scone binaryfs with SCONE_MODE=auto, as build will not have access to /dev/isgx
RUN mkdir binary-fs-dir &&  rm /usr/lib/python3.7/config-3.7m-x86_64-linux-gnu/libpython3.7m.a && \
    SCONE_MODE=auto scone binaryfs / /binary-fs-dir -v \
        --include '/usr/lib/python3.7/*' \
        --include /hello-world.py \
        --host-path=/etc/resolv.conf \
        --host-path=/etc/hosts

# Second stage: compile the binary fs
FROM registry.scontain.com/sconecuratedimages/crosscompilers:alpine AS crosscompiler

COPY --from=binary-fs /binary-fs-dir /.

RUN scone gcc ./binary_fs_blob.s ./libbinary_fs_template.a -shared -o /libbinary-fs.so


# Third stage: patch the binaryfs into the enclave executable
FROM registry.scontain.com/sconecuratedimages/apps:python-3.7.3-alpine3.10

COPY --from=crosscompiler /libbinary-fs.so /lib/libbinary-fs.so

ENV SCONE_EXTENSIONS_PATH="/lib/libbinary-fs.so"

ENV SCONE_LOG=info

CMD sh -c "python3 /hello-world.py"

Generating Binary FS Images from existing File Systems

The SCONE CLI provides the binaryfs command with which an existing directory tree can be turned into a binary fs source code file.

We illustrate the usage of this command by creating a binary fs source code image for Barbican (a python application). Within the SCONE Python image registry.scontain.com/sconecuratedimages/apps:barbican-11-alpine we execute:

scone binaryfs / /binary-fs.c \
    --include '/etc/barbican/*' \
    --include '/usr/lib/python3.7/*' \
    --include /lib/libssl.so.1.1 \
    --include /lib/libcrypto.so.1.1 \
    --include '/lib/libz.so.1*' \
    --include '/usr/lib/libbz2.so.1*' \
    --include '/usr/lib/libsqlite3.so.0*' \
    --include '/usr/lib/libev.so.4*' \
    --include '/usr/lib/libffi.so.6*' \
    --include '/usr/lib/libexpat.so.1*' \
    --include /start-barbican.py \
    --host-path=/etc/resolv.conf \
    --host-path=/etc/hosts

Info

To include files on the host file system, we use the --host-path option of binary-fs. The usage of this option is required e.g. for networking, as the binary-fs will need access to files typically managed by an underlying networking daemon. Note that these files are NOT protected by the SGX and are thereby unsafe!

Alternatively, we can copy the root directory of our binary-fs to a target directory, and create the binary-fs directly from this new directory:

# create the binary-fs root directory (which we use to create the binary-fs)
mkdir -p /my-binaryfs-root-dir

# copy all our files into the binary-fs root dir
# the root dir will be embedded into the binary-fs in `/`
cp -r /my-app /my-libs /my-binaryfs-root-dir/.

# generate the binary-fs files
SCONE_MODE=sim scone binaryfs /my-binaryfs-root-dir .

We must then compile the files created by binaryfs (binary_fs.blob, binary_fs_blob.s, libbinary_fs_template.a) using the registry.scontain.com/sconecuratedimages/crosscompilers image, as it contains the scone gcc:

scone gcc ./binary_fs_blob.s ./libbinary_fs_template.a -shared -o /libbinary-fs.so

Moving on, we move the generated libbinary-fs.so back to our original Python image. In this image, we set the .so file to be included in the enclave upon execution using the environment variable SCONE_EXTENSIONS_PATH:

SCONE_EXTENSIONS_PATH="/lib/libbinary-fs.so"

Then we are all set! When we execute the sconified executable, the runtime will include the binary-fs.

python3 /start-barbican.py

What Files to include in the File System

Typically, one must include all libraries which are not loaded by the dynamic loader, as well as any code not linked to the binary (e.g. python code files). As mentioned before, files which are changed by daemons and other host files are to be inclued with --host-path.

To find out what files to include in the binary-fs, simply run your image with strace and identify which files the binary opens:

docker run -it --rm --device /dev/isgx -e SCONE_LOG=trace --cap-add SYS_PTRACE $IMAGE sh
apk add strace
strace -o strace.log -f $EXECUTABLE
grep "open" strace.log

The output will then show which files the binary has accessed. Be sure to include these files in the binary fs to make your application function properly. Note that you do not have to include files that are already dependencies of your binary; i.e., do not include files that appear in ldd $(which your_binary).

What executables are supported

Currently, only binaries compiled as position-independent code (pie) can use the binary fs. For all other binaries, applying the binary fs may result in the binary having segmentation fault errors or the binary ignoring the binary fs. To ensure your binary is compatible, execute file $(which your_binary) and check the output if it is a pie executable. To illustrate: 

$ file $(which python3.7)
/usr/bin/python3.7: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /opt/scone/lib/ld-scone-x86_64.so.1, bad note name size 0xb57f3448, bad note name size 0xb57f3448, stripped
# pie executable -> binary fs compatible