Skip to content

SCONE Kubernetes Operator

Introduction

The SCONE Kubernetes operator facilitates a declarative description of SCONE-related custom resources. These custom resources include SCONE CAS, SCONE LAS, or the SCONE SGX Plugin. These operators have controllers to bring or keep these resources in their target state. For each of these custom resources, a custom resource definition (CRD) specifies how to define a custom resource. We introduce these CRDs in some more detail below.

We define custom resources for the following SCONE services and resources:

  • Configuration and Attestation Service (CAS):

    SCONE CAS is a central component of the SCONE infrastructure. Programs executing in enclaves connect to CAS to obtain their confidential configuration. CAS provisions this configuration only after it has verified the integrity and authenticity of the requesting enclave using remote attestation. Additionally, CAS checks that the requesting enclave is authorized to obtain the confidential configuration. One can run CAS instances on the same node as the application, the same cluster, or a different cluster. The CAS operator enables us to configure CAS policies remotely using kubectl without needing to expose CAS to any external network.

  • Local Attestation Service (LAS):

    A LAS instance must run on each Kubernetes node that supports confidential computing. Developers will not have to know about LAS as long as the SCONE operator can keep LAS running. In conjunction with CAS, it enables remote attestation of enclaves by performing a local attestation. Currently, LAS supports DCAP and EPID-based quoting enclaves. Additionally, it provides an independent SCONE quoting enclave (QE): The SCONE QE enables the decoupling of the availability of an application from Intel's attestation services. A use case of the SCONE QE is the air-gapped deployment of applications without Internet connectivity.

  • SGX Device Plugin Service (SGX Plugin):

    The SCONE SGX Plugin simplifies the deployment of confidential applications by providing any unprivileged container access to the SGX devices on hosts that support SGX. Developers will not have to know about the SGX Plugin as long as the SCONE operator can keep the SGX Plugin running. A confidential application can only run on hosts with SGX support. When running in a Kubernetes cluster, you need to make sure your application's workload is scheduled to such nodes and that the application gets access to the corresponding SGX devices. A device plugin advertises hardware resources to the Kubelet. The SCONE SGX Plugin provides access to the SGX devices. To access the SGX devices of a node, however, the Kubernetes containers usually need to run in privileged mode. The SCONE SGX Plugin will not only provide access to the SGX devices on exactly those nodes in the cluster that have support for the SGX version that you need. It will also allow your containers to run in non-privileged mode. However, the SGX Plugin must have permission to access the SGX devices.

NOTE: CAS requires both LAS and the SGX Plugin to run, and LAS requires the SGX Plugin.

We also provide controllers to ensure that SCONE policies can be submitted through custom resources in a Kubernetes cluster. Using such custom resources, we delegate the policy submission to the operator, which hides some technical details of the interactions with CAS, and allows policies to be managed through kubectl. In this case, since we do not trust Kubernetes or any cluster admins, we must protect the policies. SCONE currently provides two ways of protecting policies: Signed Policies and Encrypted Policies. In both modes, a Kubernetes admin or the controller submitting them to SCONE CAS cannot modify the contents of the policies. The following controllers and custom resources for SCONE policies are available:

  • Signed Policies:

    Signed Policies are integrity-protected. A policy can be signed through the SCONE CLI, which will use a signing key pair to sign the contents of the policy. Please note that even though the policy cannot be modified once signed, its content is still visible to anyone, which means that secrets or any sensitive information must not be included in Signed Policies. Instead, you would ask CAS to generate secrets or import these secrets from other policies.

  • Encrypted Policies:

    Encrypted Policies are integrity and confidentiality-protected. A policy can be encrypted through the SCONE CLI, which will use the public encryption key of the target SCONE CAS to encrypt the policy contents. This ensures that only the intended target SCONE CAS can decrypt the policy contents, which makes it ideal for including secrets or any sensitive information. Encrypted Policies are also signed.

In what follows, we will describe how to deploy the operator and the custom resources and configure them.

General SCONE Operator Objectives

The SCONE operator controls the lifecycle of the CAS, LAS, and SGXPlugin custom resources. Our goal is to ensure a maintenance-free operation of all three services in a Kubernetes cluster. The general objectives of the SCONE operator are as follows:

  • effortless provisioning and configuration management:

    • deploy and configure each of the SCONE services effortlessly by applying the configured manifest of the respective custom resource to the cluster,
    • permit to pull the OCI image from a private repo - instead of the default image repo, and
    • allow specification of resource limitations.
  • seamless upgrades:

    • automatically update the OCI images used to the newest available version without shutting down existing applications.
  • full lifecycle support:

    • supports full application lifecycle, including backup of the encrypted CAS database,
    • automatic failure recovery in case of node failures, and
    • disaster recovery in case a whole cluster or region becomes unavailable.
  • deep insights:

    • provide metrics of current resource usage and status of the services
    • alert the user of any problems, and
    • provide the user with access to the state of the reconciliation process
  • auto pilot:

    • automatic scaling, configuration, and scheduling tuning.
  • Kubernetes support:

    • supports different versions of Kubernetes.

Specific CAS Objectives

The functional objectives specific to the CAS operator are as follows:

  • ensure that CAS is healthy and accepting new policies and attestation requests from confidential applications, and
  • ensure that upgrades, backups of the database, and backups of the database encryption key, which ultimately allow for high-available CAS setups, are performed seamlessly.

Specific LAS Objectives

The functional objectives specific to the LAS operator are as follows:

  • ensure that LAS is deployed to all (or a subset of) SGX-enabled nodes of the cluster, thus allowing confidential applications to be attested on these nodes, and
  • ensure that there is never more than one LAS service running.

Specific SGXPlugin Objectives

The functional objectives specific to the SGXPlugin operator are as follows:

  • ensure that the SGX Plugin provides all enclaves access to an SGX device,
  • allow the SGX Plugin to be configured so that only a subset of the SGX-enabled nodes are considered when scheduling,
  • ensure that the SGX Plugin is allowing enclaves to be scheduled to all nodes where SGX is enabled (if desired), and
  • ensure there is never more than one SGX Plugin service running.