Skip to content

SCONE Binary File System

The SCONE Binary File System (Binary FS) is a mechanism with which the user can embed a file system image into an enclave binary's executable. Thereby, SGX' enclave protection mechanism is extended to the enclave's file system since the enclave's measurement (MRENCLAVE) will reflect the file system state. This is a particularly appealing solution for interpreted languages such as Python or Java where the actual program code (e.g. PyTorch or Kafka) is not part of the interpreter binary but read from byte code files in the file system. Embedding a file system into a binary means that the binary file has a dependency for a dynamic library that contains the necessary files. We create this dependency with the help of patchelf --add-needed. Such an approach provides flexibility of patching an existing binary (e.g. Python or Java) rather than a long recompilation of this binary with an object file. Note that, 1) modifications the program does to the binary fs are not persistent, after each start the binary fs is in its initial state, and 2) the binary fs replaces the entire file system, i.e. an enclave with binary fs won't be able to read or write from the host's file system. You can utilize SCONE volumes to persistently store data on the host's file system.

TL;DR

A fully functioning single Dockerfile which runs a Python hello_world.py program using the binary-fs could look like this:

# First stage: apply the binary-fs
FROM registry.scontain.com:5050/sconecuratedimages/apps:python-3.7.3-alpine3.10 AS binary-fs

RUN echo "print('Hello World!')" > /hello-world.py

# remove object archive as very large and not necessary for executing the python program
# & apply scone binaryfs with SCONE_MODE=auto, as build will not have access to /dev/isgx
RUN rm /usr/lib/python3.7/config-3.7m-x86_64-linux-gnu/libpython3.7m.a && \
    SCONE_MODE=auto scone binaryfs / /binary-fs.c -v \
        --include '/usr/lib/python3.7/*' \
        --include /hello-world.py

# Second stage: compile the binary fs
FROM registry.scontain.com:5050/sconecuratedimages/crosscompilers:alpine AS crosscompiler

COPY --from=binary-fs /binary-fs.c /.

RUN scone gcc /binary-fs.c -O0 -shared -o /libbinary-fs.so


# Third stage: patch the binary-fs into the enclave executable
FROM registry.scontain.com:5050/sconecuratedimages/apps:python-3.7.3-alpine3.10

COPY --from=crosscompiler /libbinary-fs.so /.

RUN apk add --no-cache patchelf && \
    patchelf --add-needed libbinary-fs.so `which python3` && \
    apk del patchelf


ENV SCONE_HEAP=512M
ENV SCONE_LOG=debug
ENV LD_LIBRARY_PATH="/"

CMD sh -c "python3 /hello-world.py"

Basic Usage

SCONE Binary FS is implemented as a SCONE runtime extension with which users can modify certain behavior of the SCONE runtime. The particular interface that has to be implemented for the Binary FS (as all other SCONE runtime extension interfaces) is described in the scone_rt_ext.h header file of the SCONE cross compiler installation that can be accessed in the cross-compiler container with cat /opt/scone/cross-compiler/x86_64-linux-musl/include/scone_rt_ext.h.

This is the source code of a "hello world" example of a binary fs SCONE runtime extension:

#include <stdlib.h>
#include <scone_rt_ext.h>

binary_fs_config_t fs_config = {
    .binary_fs_file_t_size = sizeof(binary_fs_file_t),
};

binary_fs_file_t *files[] = {
    &(binary_fs_file_t){
        .path = "/farewell/english",
        .content = "Goodbye",
        .size = 8,
    },
    NULL
};

scone_rt_hook_status_t scone_rt_hook_binary_fs(binary_fs_interface_version_t *version, binary_fs_config_t *config, binary_fs_file_t **file[]) {
    *version = BINARY_FS_VERSION_V1;
    *config = fs_config;
    *file = files;

    return SCONE_HOOK_STATUS_SUCCESS;
}

It has to be compiled and linked into the enclave binary to make use of it:

# Compilation of binary fs
$ scone gcc ./binary-fs.c -shared -o ./binary-fs.so
# Linking into enclave binary
$ scone gcc ./print_file.c ./binary-fs.so -o print_file

# Printing file that is part of the binary fs in the host file system
$ cat /farewell/english
cat: /farewell/english: No such file or directory
# Printing the file from the enclave
$ ./print_file /farewell/english
Goodbye

# Printing file that exists in the host file system
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"
# But not in the binary fs
$ ./print_file /etc/lsb-release
No such file or directory

Generating Binary FS Images from existing File Systems

The SCONE CLI provides the binaryfs command with which an existing directory tree can be turned into a binary fs source code file.

We illustrate the usage of this command by creating a binary fs source code image for Barbican (a python application). Within the SCONE Python image registry.scontain.com:5050/sconecuratedimages/apps:barbican-11-alpine we execute:

scone binaryfs / /binary-fs.c \
    --include '/etc/barbican/*' \
    --include '/usr/lib/python3.7/*' \
    --include /lib/libssl.so.1.1 \
    --include /lib/libcrypto.so.1.1 \
    --include '/lib/libz.so.1*' \
    --include '/usr/lib/libbz2.so.1*' \
    --include '/usr/lib/libsqlite3.so.0*' \
    --include '/usr/lib/libev.so.4*' \
    --include '/usr/lib/libffi.so.6*' \
    --include '/usr/lib/libexpat.so.1*' \
    --include /start-barbican.py

We must then compile the binary-fs.c file using the registry.scontain.com:5050/sconecuratedimages/crosscompilers image, as it contains the scone gcc:

scone gcc /binary-fs.c -O0 -shared -o /libbinary-fs.so

Moving on, we move the generated libbinary-fs.so back to our original Python image. In this image, we link the .so file to the enclave binary using patchelf:

apk add patchelf
patchelf --add-needed libbinary-fs.so `which python3`

Currently, patchelf does not update the signature of the binary, although it changes the binary's contents. Thus, running the binary without further environment modification will produce an error. To ensure that we update the corresponding signature, we must set the SCONE_HEAP environment variable to a different value than it was before. The setting of this variable will prompt SCONE to recompute the signature of the executable. The enclave executable will then function properly:

SCONE_HEAP=512M python3 /start-barbican.py

What Files to include in the File System

To find out what files to include in the binary-fs, simply run your image with strace and identify which files the binary opens:

docker run -it --rm --device /dev/isgx -e SCONE_LOG=trace --cap-add SYS_PTRACE $IMAGE sh
apk add strace
strace -o strace.log -f $EXECUTABLE
grep "open" strace.log

The output will then show which files the binary has accessed. Be sure to include these files in the binary fs to make your application function properly. Note that you do not have to include files that are already dependencies of your binary; i.e., do not include files that appear in ldd $(which your_binary).

Possible Networking Problems

Note that the binary-fs will not have access to /etc/ files which are only available at deployment on e.g. a cluster. These files may include /etc/resolv.conf, which your service needs to access the DNS. If you are deploying your service with binary-fs to a Kubernetes cluster, include the following injection file in your policy for resolv.conf:

images:
  - name: your_image
    injection_files:
      ...
      # Network files
      - path: /etc/resolv.conf
        content: |
           nameserver 10.96.0.10
           search sgx-scone.svc.cluster.local svc.cluster.local cluster.local
           options ndots:5

This addresses the Cluster's CoreDNS service, with which your service will be able to address other services within the Cluster.

What executables are supported

Currently, only binaries compiled as position-independent code (pie) can use the binary fs. For all other binaries, applying the binary fs may result in the binary having segmentation fault errors or the binary ignoring the binary fs. To ensure your binary is compatible, execute file $(which your_binary) and check the output if it is a pie executable. To illustrate:

$ file $(which python3.7)
/usr/bin/python3.7: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /opt/scone/lib/ld-scone-x86_64.so.1, bad note name size 0xb57f3448, bad note name size 0xb57f3448, stripped
# pie executable -> binary fs compatible