Skip to content

Kubernetes SGX Plugin (sgxdevplugin)

Prerequisites

Background

Your Kubernetes cluster might be heterogeneous, i.e., it might contain nodes with different capabilities. Hence, some of the nodes might have SGX version 1 support and some might have full SGX version 2 support and some might have partial SGX version 2 support and some servers might have no SGX support at all. With SCONE, you can run the same binary on version 1 or version 2 hosts. Depending on the version of your CPU and your SGX driver, you might see devices /dev/sgx (DCAP) or /dev/isgx (IAS) or ... . You should not - and you do not - need to care about about these details.

SCONE will find the right SGX device and will adjust its behavior depending on the kind of CPU you are running on. When scheduling your confidential application on a cluster, you ideally want to just specify that you need SGX support. Of course, in addition to this basic scheduling on a SGX-capable node, you can use all the niceties of Kubernetes, like affinity and tolerations, to optimize the scheduling.

To access the SGX device in a Kubernetes cluster, you might need to run the containers in privileged mode. Of course, one wants to avoid running applications in privileged mode. Our Kubernetes SGX plugin allows you to run containers accessing the SGX device without requiring privileged mode. Note, however, that the plugin has to run as a privileged container itself.

If you prefer to run without this plugin, you can do so as we describe in this tutorial. Note, however, that our sconeapps helm charts expect this Kubernetes SGX plugin.

Usage

We provide a plugin for Kubernetes that permits containers to access the SGX device without having privileged access. You will just need to specify that your container requires access to a SGX device. However, you need to give this container access to RAWIO.

You can specify that a container (MariaDB) of your Pod db requires access to SGX as follows:

apiVersion: v1
kind: Pod
metadata:
  name: db
spec:
  containers:
    - name: MariaDB
      image: registry.scontain.com:5050/sconecuratedimages/mariadb:production
      securityContext:
        capabilities:
          add: ["SYS_RAWIO"]
      resources:
        limits:
          sgx.k8s.io/sgx: 1   # Requires to some sgx device like /dev/isgx or /dev/sgx or successor

With sgx you say that any SGX device is fine for your applications. This should be the default with SCONE-based applications.

In case you need to run on a machine which has a DCAP-based attestation, you can specify as follows:

        limits:
          sgx.k8s.io/dcap: 1   # Requires DCAP-based attestation

In case you need to run on a machine which has an IAS-based attestation, you can specify as follows:

        limits:
          sgx.k8s.io/ias: 1   # Requires IAS-based attestation

Deploying Kubernetes SGX Plugin

The sconeapps/sgx_plugin chart will deploy the Kubernetes SGX plugin in your cluster. You can just execute:

helm install sgxdevplugin sconeapps/sgxdevplugin

This starts the Kubernetes SGX plugin in the default Kubernetes namespace. This will limit the number of SGX devices per node to 20 (default). If you want to run more enclaves per node, please increase the the value.

To increase the maximum allocatable SGX devices per node, please increase the value of maxSGXPods. You can set this value by executing:

helm install sgxdevplugin sconeapps/sgxdevplugin  --set maxSGXPods=100

If the plugin is already executing, you can change the value as follows:

helm upgrade  sgxdevplugin sconeapps/sgxdevplugin --set maxSGXPods=100

Checking Status of SGX devices

You can check the number of allocatable SGX devices on all nodes of your Kubernetes cluster as follows:

export NODES=`kubectl get node --output=name`
for N in $NODES; do  echo "$N:" ; kubectl get $N -o jsonpath='{.status.allocatable}' ; echo ;  done

If the number of SGX devices is too small, increase the number of allocatable SGX devices as explained above.