Kubernetes SGX Plugin (sgxdevplugin)

The easiest way to deploy SGX plugin is to use the SCONE Operator.

Background

Prerequisites

A Kubernetes cluster
Helm setup was performed

Your Kubernetes cluster might be heterogeneous, i.e., it might contain nodes with different capabilities. Hence, some of the nodes might have SGX version 1 support and some might have full SGX version 2 support and some might have partial SGX version 2 support and some servers might have no SGX support at all. With SCONE, you can run the same binary on version 1 or version 2 hosts. Depending on the version of your CPU and your SGX driver, you might see devices /dev/sgx_enclave, /dev/sgx/sgx_enclave (legacy), /dev/isgx (legacy) or /dev/sgx (legacy). You should not - and you do not - need to care about about these details.

SCONE will find the right SGX device and will adjust its behavior depending on the kind of CPU you are running on. When scheduling your confidential application on a cluster, you ideally want to just specify that you need SGX support. Of course, in addition to this basic scheduling on an SGX-capable node, you can use all the niceties of Kubernetes, like affinity and tolerations, to optimize the scheduling.

To access the SGX device in a Kubernetes cluster, you might need to run the containers in privileged mode. Of course, one wants to avoid running applications in privileged mode. Our Kubernetes SGX plugin allows you to run containers accessing the SGX device without requiring privileged mode. Note, however, that the plugin has to have sufficient permission to access the SGX device itself.

If you prefer to run without this plugin, you can do so as we describe in this tutorial. Note, however, that our sconeapps helm charts expect this Kubernetes SGX plugin.

Usage

We provide a plugin for Kubernetes that permits containers to access the SGX device without having privileged access. You will just need to specify that your container requires access to an SGX device. However, you need to give this container access to RAWIO.

You can specify that a container (MariaDB) of your Pod db requires access to SGX as follows:

apiVersion: v1
kind: Pod
metadata:
  name: db
spec:
  containers:
    - name: MariaDB
      image: registry.scontain.com/sconecuratedimages/mariadb:production
      securityContext:
        capabilities:
          add: ["SYS_RAWIO"]
      resources:
        limits:
          sgx.intel.com/enclave: 1   # Requires to some sgx device like /dev/isgx or /dev/sgx or successor

With sgx you say that any SGX device is fine for your applications. This should be the default with SCONE-based applications.

In case you need to run on a machine which has a DCAP-based attestation, you can specify as follows:

        limits:
          sgx.k8s.io/dcap: 1   # Requires DCAP-based attestation

In case you need to run on a machine which has an IAS-based attestation, you can specify as follows:

        limits:
          sgx.k8s.io/ias: 1   # Requires IAS-based attestation

Deploying Kubernetes SGX Plugin

The sconeapps/sgx_plugin chart will deploy the Kubernetes SGX plugin in your cluster. You can just execute:

helm install sgxdevplugin sconeapps/sgxdevplugin

This starts the Kubernetes SGX plugin in the default Kubernetes namespace. This will limit the number of SGX devices per node to 20 (default). If you want to run more enclaves per node, please increase the the value.

To increase the maximum allocatable SGX devices per node, please increase the value of maxSGXPods. You can set this value by executing:

helm install sgxdevplugin sconeapps/sgxdevplugin  --set maxSGXPods=100

If the plugin is already executing, you can change the value as follows:

helm upgrade  sgxdevplugin sconeapps/sgxdevplugin --set maxSGXPods=100

Checking Status of SGX devices

You can check the number of allocatable SGX devices on all nodes of your Kubernetes cluster as follows:

export NODES=`kubectl get node --output=name`
for N in $NODES; do  echo "$N:" ; kubectl get $N -o jsonpath='{.status.allocatable}' ; echo ;  done

If the number of SGX devices is too small, increase the number of allocatable SGX devices as explained above.

Exposed resources

Resource	What you get
`sgx.intel.com/enclave`, `sgx.k8s.io/sgx`	Any available SGX device: IAS (/dev/isgx) or DCAP (/dev/sgx or /dev/sgx_enclave)
`sgx.k8s.io/ias`	IAS devices: /dev/isgx
`sgx.k8s.io/dcap`	DCAP devices: /dev/sgx or /dev/sgx_enclave. Provisioning devices are also mapped

Labeling SGX-capable nodes

If labelNodes is set to true (default), the SGX plugin will label the SGX-capable nodes with the label sgx.intel.com/capable="true". This allows the SGX-capable nodes to be selected through the label.

For example: SCONE LAS, which is a DaemonSet, will be deployed to all cluster nodes, as it is the default DaemonSet behavior. However, since LAS relies on SGX devices to run, if you have a cluster with SGX and non-SGX nodes, it makes no sense to deploy it to non-SGX nodes. We could leverage the label sgx.intel.com/capable="true" to select the right nodes for LAS:

Parameters

A complete list of parameters this chart supports.

Parameter	Description	Default
`maxSGXPods`	Maximum allocatable SGX devices per node	`110`
`labelNodes`	Whether the plugin should label SGX-capable nodes with `sgx.intel.com/capable="true"`	`true`
`image`	SGX device plugin image	`registry.scontain.com/sconecuratedimages/kubernetes:k8s-sgx-plugin`
`imagePullPolicy`	SGX device plugin image pull policy	`Always`
`nodeSelector`	Node labels for pod assignment (this value is evaluated as a template)	`{}`