Skip to content

SCONE File Protection

This section describes the low-level SCONE file encryption interface to encrypt the root filesystem of an image. In many cases, one might want to use a SCONE volume instead:

  • a SCONE volume is typically mapped on a Kubernetes volume with access modes ReadWriteOnce or ReadOnlyMany
  • a SCONE volume is automatically encrypted, i.e., no need to perform the fspf operations described in this section
  • a SCONE volume - as a Kubernetes volume - exists only once and SCONE CAS tracks its confidentiality and integrity including freshness.

Concepts

SCONE supports the transparent encryption and/or authentication of files of the root file system of a container / image. By transparent, we mean that there are no application code changes needed to support this. The underlying idea of SCONE file protection is that a user specifies that each file is either:

  • authenticated, i.e., SCONE checks that the content was not modified by some unauthorized entity,

  • encrypted, i.e., the confidentiality is protected by encryption. Encrypted files are always authenticated, or

  • not-protected, i.e. SCONE reads and write the files without any extra protection mechanisms. For example, you might use not-protected if your application already encrypts its files or if you need direct access to devices.

Marking all files individually as either authenticated, encrypted, or not-protected would not be very practical. Hence, we support to partition the filesystem into regions: regions do not overlap and each file belongs to exactly one region.

A region is defined by a path. For example, region / is the root region and you could, for example, specify that all files in region / must be authenticated. You can define a second region, for example, region /data/db and that this region is encrypted.

Each file belongs to exactly one region: it belongs to the region that has the longest common path prefix with this file. For example, file /etc/db.conf would belong, in this case, to region / and file /data/db/table.db would belong to region /data/db.

SCONE supports ephemeral regions: files are stored in main memory outside of the enclave. Since the main memory is not protected, we recommend that an ephemeral regions is either authenticated or encrypted. When a program starts, all its ephemeral regions are empty. The only way to add files to an ephemeral region is by the application writing to this region. All files in an ephemeral region are lost when the application exits.

All files that need to be persistent should be stored in a non-ephemeral region instead. We refer to this as a kernel region. For each region, you need to specify if the region is either ephemeral or kernel.

Each region belongs to one of the following six classes:

{ephemeral | kernel} X {not-protected | authenticated | encrypted }

Example

Sometimes, we might only need to protect the files that are passed to a container via some volume. In this case, it would be sufficient that the volume is either authenticated or encrypted.

Let us demonstrate this via a simple example in which we pass an encrypted volume to a container. We create this encrypted volume in our local filesystem (in directory volume) and we will later mount this in the container as /data. The original (non-encrypted) files are stored in directory data-original.

> mkdir -p volume
> mkdir -p data-original

Let's write some files in the data-original directory:

cat > data-original/hello.txt << EOF
Hello World
EOF
cat > data-original/world.py << EOF
f = open('/data/hello.txt', 'r')
print str(f.read())
EOF

Let's check that volume is empty and we print the hash values of the two files in data-original:

> ls volume
> shasum data-original/*
648a6a6ffffdaa0badb23b8baf90b6168dd16b3a  data-original/hello.txt
deda99d44e880ea8f2250f45c5c20c15d568d84c  data-original/world.py

Now, we start the SCONE crosscompiler in a container to create the encrypted volume:

> docker run -it -v "$PWD/volume:/data" -v "$PWD/data-original:/data-original" registry.scontain.com/sconecuratedimages/crosscompilers

File System Protection File

All the metadata required for checking the consistency of the files is stored in a file system protection file, or, short fspf. SCONE supports multiple *fspf*s.

Let's start with a simple example with a single fspf. The fspf file is created via command scone fspf create and let us name this file fspf.pb. We execute the following commands inside the container (as indicated by the $ prompt):

$ cd /data
$ scone fspf create fspf.pb
Created empty file system protection file in fspf.pb. AES-GCM tag: 0e3da7ad62f5bc7c7bb08c67b16f2423

We can now split the file system in regions, a region is a subtree. You can add regions to a fspf with the help of command scone fspf addr.

Each region has exactly one of the following properties:

  • authenticated: the integrity of files is checked, i.e., any unauthorized modification of this file is detected and results in a reading error inside of the enclave. Specify command line option --authenticated.
  • encrypted: the confidentiality and integrity of files is protected, i.e., encrypted always implies that the files are also authenticated. Specify command line option --encrypted.
  • not-protected: files are neither authenticated nor encrypted. Specify command line option --not-protected.

File system changes of containers are typically ephemeral in the sense that file updates are lost when a container terminates. When specifying option --ephemeral, files in this region are not written to disk, the are written to an in memory file system instead.

Say for now, that by default we do not protect files and we want to read files and write back changed files to the file system. To do so, we define that the root tree is --kernel as well as --not-protected:

$ scone fspf addr fspf.pb / --kernel / --not-protected
Added region / to file system protection file fspf.pb new AES-GCM tag: dd961af10b5aaa5cb1044c35a3f42c84

You need to specify root /

In case you define more than one region, you always need to define a root region "/".

Let us add another region /data that should be encrypted and persisted. To encrypt the files, we specify option --encrypted. We specify option --kernel followed by a path (here, also /data) to request that files in this region are written to the kernel file system into directory /data.

$ scone fspf addr fspf.pb /data --encrypted --kernel /data
Added region /data to file system protection file fspf.pb new AES-GCM tag: 8481369d3ffdd9b6aeb30d044bf5c1c7

The encryption key for a file is chosen at random and stored in fspf.pb. We use the Intel random number generator RdRand to generate the key. The default key length of a region is 32 bytes. Alternatives are key length of 16 and 24 bytes. These can be selected via option --key-length 16 and --key-length 24 when creating a region with command scone fspf addr.

Now, that we defined the regions, i.e., / and /data, we can add files to region /data. Let's just add all files in /data-original, encrypt these and write the encrypted files to /data. Note, the first /data argument specifies the protection region that determines the protection policy. The second, specifies where the encrypted files will be stored. That is, the command iterates over and reads the existing files in /data-original and encrypts them. The encrypted file content is written into the directory /data while the protection metadata of the individual files is added to the fsfp.pb file.

$ scone fspf addf fspf.pb /data /data-original /data
Added files to file system protection file fspf.pb new AES-GCM tag: 39a268166e628cf76e3fca80aa2d4f63

Note that if the /data region would have been only authenticated and not encrypted, the tool does not need to write out any (encrypted) files. It will only add the file names and the checksums (tags) of the files located in /data-original to the fspf.pb file. Thus, you could drop the last argument in this case.

Coming back to the above example, we can now compare the hash values of the original files and the encrypted files:

$ shasum /data/*
87fd97468024e3d2864516ff5840e15d9615340d  /data/fspf.pb
31732914910f4a08b9832c442074b0932915476c  /data/hello.txt
8d07f3f576785c373a5e70e8dbcfa8ee06ca6d0c  /data/world.py
$ shasum /data-original/*
648a6a6ffffdaa0badb23b8baf90b6168dd16b3a  /data-original/hello.txt
deda99d44e880ea8f2250f45c5c20c15d568d84c  /data-original/world.py

The fspf itself is not yet encrypted. We encrypt this file via command scone fspf encrypt fspf.pb

$ scone fspf encrypt fspf.pb > /data-original/keytag
We store the random encryption key as well as the tag of file fspf.pb in file /data-original/keytag.

We introduce a very simple program that reads the two files:

$ cat > example.c << EOF
#include <stdio.h>
#include <stdlib.h>

void printfile(const char* fn) {
    FILE *fp = fopen(fn, "r");
    char c;
    while((c=fgetc(fp))!=EOF){
        printf("%c",c);
    }
    fclose(fp);
}

int main() {
    printfile("/data/hello.txt");
    printfile("/data/world.py");
}
EOF

Let's crosscompile this program:

scone gcc example.c -o example

Executing this program results in an output like this:

$./example
R??C?
    q?z??E??|Ю?}ü ?o
$??!rga??·*`?????????Gw?

We need to activate the file system shield via environment variables by setting the location of the file system protection file (in SCONE_FSPF), the encryption key of the file (in SCONE_FSPF_KEY) and the tag of the fspf (in SCONE_FSPF_TAG). We can extract the encryption key as well as the tag of fspf.pb from file /data-original/keytag:

$ export SCONE_FSPF_KEY=$(cat /data-original/keytag | awk '{print $11}')
$ export SCONE_FSPF_TAG=$(cat /data-original/keytag | awk '{print $9}')
$ export SCONE_FSPF=/data/fspf.pb

We can now execute this program again:

$ ./example
Hello World
f = open('/data/hello.txt', 'r')
print str(f.read())

Variables SCONE_FSPF_KEY, SCONE_FSPF_TAG and SCONE_FSPF should only be set manually for debugging since they cannot securely be passed in this way to programs running inside enclaves. To securely pass environment variables, please use CAS.

Python

Let's try a similar approach for Python. In the above example, we encrypted a Python program. Let's try to execute this encrypted program that accesses an encrypted file:

docker run -it -v "$PWD/volume:/data" registry.scontain.com/sconecuratedimages/apps:python-2.7-alpine3.6 bash

The files /data/world.py and /data/hello.txt are encrypted:

$ cat /data/world.py
?=??J??0?6+?Q?nKd?*N,??.?G???????R?cO?t?y??>f?

Let's activate the file shield:

$ export SCONE_FSPF_KEY=... extract from data-original/keytag ...
$ export SCONE_FSPF_TAG=... extract from data-original/keytag ...
$ export SCONE_FSPF=/data/fspf.pb

We can now run the encrypted world.py program with the the Python interpreter:

SCONE_HEAP=100000000 SCONE_ALPINE=1 SCONE_VERSION=1 /usr/local/bin/python /data/world.py
export SCONE_QUEUES=1
...
Hello World

Protecting the Root Region

Note that in the above example, Python will not be permitted to load dynamic libraries outside of the protected directory /data: a dynamic library must reside in either an authenticated or an encrypted region. To deal with this, we must define one or more authenticated or encrypted file regions that contain the dynamic libraries.

Let us show how to authenticate all files in region /

$ scone fspf addr fspf.pb / --kernel / --authenticated

We need to add all files that our application might access. Often, these files in the root region might be defined in some container image. Let's see how we can add these files to our region /.

Adding files from an existing container image

We show how to add a subset of the files of container image registry.scontain.com/sconecuratedimages/apps:python-2.7-alpine3.6 to our root region. To do so, we ensure that we have the newest images:

> docker pull registry.scontain.com/sconecuratedimages/apps:python-2.7-alpine3.6
> docker pull registry.scontain.com/sconecuratedimages/crosscompilers

How can we add all files in a container to the fspf? One way to do so requires to run Docker inside of a Docker container. To be able to do so, we need to permit our outermost docker container to have access to /var/run/docker.sock:

> docker run -it -v /var/run/docker.sock:/var/run/docker.sock -v "$PWD/volume:/data" -v "$PWD/data-original:/data-original" registry.scontain.com/sconecuratedimages/crosscompilers

Let us ensure that Docker is installed in this container:

apt-get update
apt-get install -y docker.io

Now, we want to add all files of some target container. In our example, this is an instance of image registry.scontain.com/sconecuratedimages/apps:python-2.7-alpine3.6. We ensure that we pulled the latest image before we start the container:

CONTAINER_ID=`docker run -d registry.scontain.com/sconecuratedimages/apps:python-2.7-alpine3.6 printf OK`

We can now copy all files from this container into a new directory rootvol:

$ cd
$ mkdir -p rootvol
$ docker cp $CONTAINER_ID:/ ./rootvol

Now that we have a copy of the files, we should not forget to garbage collect this container:

docker rm $CONTAINER_ID

Let's remove some directories that we do not want our program to access, like for example, /dev:

$ rm -rf rootvol/dev rootvol/proc rootvol/bin rootvol/media rootvol/mnt rootvol/usr/share/X11 rootvol/usr/share/terminfo rootvol/optrootvol/usr/include/c++/ rootvol/usr/lib/tcl8.6 rootvol/usr/lib/gcc rootvol/opt rootvol/sys rootvol/usr/include/c++

Now, we create a root fspf:

$ scone fspf create fspf.pb
$ scone fspf addr fspf.pb / --kernel / --authenticated
$ scone fspf addf fspf.pb / ./rootvol /
$ scone fspf encrypt fspf.pb > keytag

We can now create a new container image with this file system protection file using this Dockerfile

$ cat > Dockerfile << EOF
FROM registry.scontain.com/sconecuratedimages/apps:python-2.7-alpine3.6
COPY fspf.pb /
EOF
$ docker build -t registry.scontain.com/sconecuratedimages/apps:python-2.7-alpine3.6-authenticated .

We can run a container as follows: 

$ docker run -it registry.scontain.com/sconecuratedimages/apps:python-2.7-alpine3.6-authenticated sh

Let us activate the file shield:

$ export SCONE_FSPF_KEY=... extract from data-original/keytag ...
$ export SCONE_FSPF_TAG=... extract from data-original/keytag ...
$ export SCONE_FSPF=/fspf.pb

Let's run python with authenticated file system:

SCONE_HEAP=1000000000 SCONE_ALLOW_DLOPEN=2  SCONE_ALPINE=1 SCONE_VERSION=1 /usr/local/bin/python

Checking the File System Shield

Let's us check the file shield by creating a new python program (helloworld-manual.py) in side of a python container:

> docker run -i registry.scontain.com/sconecuratedimages/apps:python-2.7-alpine3.6-authenticated sh
$ cat > helloworld-manual.py << EOF
print "Hello World"
EOF

When we switch on the file shield, the execution of this program inside the enclave will fail: since this file was not part of the original file system, the file system shield will prevent accessing this file.

$ export SCONE_FSPF_KEY=... extract from data-original/keytag ...
$ export SCONE_FSPF_TAG=... extract from data-original/keytag ...
$ export SCONE_FSPF=/fspf.pb
$ SCONE_HEAP=1000000000 SCONE_ALLOW_DLOPEN=2  SCONE_ALPINE=1 SCONE_VERSION=1 /usr/local/bin/python helloworld-manual.py
(fails)

We can, however, add a new file via programs that have access to the key of the fspf. We can, for example, write a python program to add a new python program to the file system.

By default, we disable that the root fspf is updated. We can enable updates by setting environment variable SCONE_FSPF_MUTABLE=1. We plan to permit updates of the root fspf by default in the near future (i.e., we will remove variable SCONE_FSPF_MUTABLE=1).

$ SCONE_HEAP=1000000000 SCONE_FSPF_MUTABLE=1 SCONE_ALLOW_DLOPEN=2  SCONE_ALPINE=1 SCONE_VERSION=1 /usr/local/bin/python  << PYTHON
f = open('helloworld.py', 'w')
f.write('print "Hello World"\n')
f.close()
PYTHON```

The tag of the file system protection file is now changed. We can determine the new TAG with the help of command scone fspf show:

$ export SCONE_FSPF_TAG=$(scone fspf show --tag /fspf.pb)

Now, we can run the new helloworld.py:

$ SCONE_HEAP=1000000000 SCONE_ALLOW_DLOPEN=2  SCONE_ALPINE=1 SCONE_VERSION=1 /usr/local/bin/python helloworld.py
...
Hello World

Extended Example

To learn how to use multiple file system protection files, please have a look at the following screencast.

Below is the script that is executed in the screencast:

docker run -it -v $PWD:/mnt registry.scontain.com/sconecuratedimages/crosscompilers

mkdir -p /example
mkdir -p /mnt/authenticated/
mkdir -p /mnt/encrypted/
cd /example
mkdir -p .original

scone fspf create fspf.pb

# add protection regions
scone fspf addr fspf.pb / -e --ephemeral
scone fspf addr fspf.pb /mnt/authenticated -a --kernel /mnt/authenticated
scone fspf addr fspf.pb /mnt/encrypted -e --kernel /mnt/encrypted

# add files

# enclave program should expect the files (directories) found by the client in ./original in /mnt/authenticated
scone fspf addf fspf.pb /mnt/authenticated ./original

# enclave program should expect the files (directories) found by the client in ./original in encrypted form in /mnt/encrypted
# the client will write the encrypted files to ./mnt/encrypted
scone fspf addf fspf.pb /mnt/encrypted ./original ./mnt/encrypted
KEYTAG=`scone fspf encrypt fspf.pb`
export SCONE_FSPF_KEY=`echo $KEYTAG | awk '{print $11}'`
export SCONE_FSPF_TAG=`echo $KEYTAG | awk '{print $9}'`

echo "SCONE_FSPF_KEY=${SCONE_FSPF_KEY} ; SCONE_FSPF_TAG=${SCONE_FSPF_TAG}"
cat > example.c << EOF
#include <stdio.h>

int main() {
    FILE *fp = fopen("/mnt/authenticated/hello", "w");
    fprintf(fp, "hello world\n");
    fclose(fp);

    fp = fopen("/mnt/encrypted/hello", "w");
    fprintf(fp, "hello world\n");
    fclose(fp);
}
EOF

scone gcc example.c -o sgxex
cat > /etc/sgx-musl.conf << EOF
Q 4
e -1 0 0
s -1 0 0
e -1 1 0
s -1 1 0
e -1 2 0
s -1 2 0
e -1 3 0
s -1 3 0
EOF
SCONE_FSPF=fspf.pb ./sgxex
cat /mnt/authenticated/hello
cat /mnt/encrypted/hello
cat > cat.c << EOF
#include <stdio.h>

int main() {
    char buf[80];
    FILE *fp = fopen("/mnt/authenticated/hello", "r");
    fgets(buf, sizeof(buf), fp);
    fclose(fp);
    printf("read: '%s'\n", buf);

    fp = fopen("/mnt/encrypted/hello", "r");
    fgets(buf, sizeof(buf), fp);
    fclose(fp);
    printf("read: '%s'\n", buf);
}
EOF

scone gcc cat.c -o native_cat
./native_cat
scone gcc cat.c -o sgxcat
SCONE_FSPF=fspf.pb ./sgxcat