Skip to content

CAS Backup Controller

This Kubernetes controller ensures that other cluster nodes have a registered CAS backup. This enables CAS to be restarted on a different node of a Kubernetes cluster. Note that the CAS Backup Controller does NOT backup the CAS database: You need to explicitly back up the Kubernetes volume used by CAS.


The CAS Backup Controller can run either locally (as a standalone container) or in a Kubernetes cluster. It requires two assets to run and register backups:

  1. Access to the Kubernetes cluster via a Kubernetes configuration file (whose default location is ~/.kube/config), if running locally, or via a ServiceAccount, if running on a cluster.
  2. The primary CAS owner identity via a SCONE CLI configuration file. The owner identity is created during the provisioning process of the primary CAS, and it is stored in the SCONE CLI configuration file (whose default location is ~/.cas/config).

Please note that the primary CAS must have persistence enabled and be configured appropriately.

Quick start

For an end-to-end example, please check the Quick Start guide.


CAS encrypts its database with an encryption key derived from the SGX private Seal Key, which is unique to the particular platform/machine. Although very secure, one practical effect of having the records encrypted this way is that they cannot be opened on other machines (since the private Seal Key is different). This is a limitation for production systems, mainly if they are managed by orchestration tools, such as Kubernetes, which assume the workloads to be fault-tolerant and have their deployment decoupled from specific infrastructure nodes.

To overcome this limitation, the user can securely register backup encryption keys through the CAS Backup feature, allowing the encrypted records to be opened on other machines.

The CAS Backup Controller automatically registers backup encryption keys from all the nodes in a Kubernetes cluster. This allows CAS to open its encrypted database even when migrated to another node in the cluster (e.g., to recover from a node failure).

Note that one needs to ensure that one always has at least one backup registered. Consider that the nodes of a Kubernetes cluster running on a set of servers S1. The backup controller registers backups for each server in S1. If one shuts down all nodes of a Kubernetes cluster, one needs to ensure when the cluster is started again on a new set of servers S2 that S1 and S2 have at least one server in common. If this cannot be guaranteed, one might need to register a backup on an additional server B such that one can use B to recover the CAS database.

How it works

The Controller relies on a SCONE CLI configuration file with an owner identity. This configuration file can be provided locally (--config path/to/config.json) or injected via a session (--session session_name/service_name). Since the SCONE CLI configuration holds sensitive information that could allow someone to impersonate the CAS owner, using a session is preferred.

The Kubernetes configuration is retrieved using load_kube_config and load_incluster_config functions to load the cluster configuration from the environment (be it a local KUBECONFIG file or a ServiceAccount token, if inside the cluster).

The Controller loop scans the nodes periodically, indexing them by their UID. Master nodes are skipped. If a selector label is defined (e.g., export SELECTOR_LABEL=""), all nodes that do not match the selector are also skipped.

Selected nodes are added to an internal, in-memory data structure to keep track of which nodes have registered backups and which nodes are still missing. The Controller skips selected nodes that are not schedulable at the moment until the next loop.

Ephemeral backup CAS resources are then deployed to selected nodes that are schedulable. When the CAS Pod is reported as "Ready", the Controller will try to register them as a backup for the primary CAS. If this operation fails, the error will appear in the Controller output, and the Controller will continue the loop (failed registrations are attempted again in the following loops). Regardless of the outcome, ephemeral backup CAS resources are deleted right after the registration operation. If it fails, new ephemeral backup CAS resources will be created in the next loop.

Please note that the primary CAS requires all registered backup CAS to have a TCB (Trusted Computing Base) in an equal or better state.

New cluster nodes will be automatically included in the loop. No action is taken for deleted cluster nodes.

Detecting hardware changes

Since the CAS encryption key is derived from the underlying hardware (i.e., the SGX seal key), we have to detect hardware changes and register backups with the new keys again. Detecting such changes is challenging in some scenarios, especially when the user does not control the infrastructure (e.g., VMs in public cloud providers).

The Controller implements two strategies to solve this issue. You can specify a label selector for LAS pods (through environment variable LAS_SELECTOR_LABEL), allowing the Controller to keep track of their public keys on each node. Since the LAS public key is also derived from the SGX seal key, it is unique per physical machine. Any LAS public key change causes the cluster node to have a backup registered again. If the cluster does not have any LAS pods, or if any issue happens when fetching the public keys, the Controller will fall back to checking the boot_id of the cluster nodes, as exposed by Kubernetes. Any change in the boot_id (indicating a node restart) will cause the cluster node to have a backup registered again.

Remote attestation

To enable remote attestation, specify --session session/service argument to the Controller (e.g., --session backup_controller/register).

The submitted session should contain the SCONE CLI configuration file with an owner identity. This configuration file is obtained through the provisioning process of the primary CAS.

This is the recommended way of running the Controller in a Kubernetes cluster (or in any potentially untrusted infrastructure, really), because you can inject the owner configuration file securely through CAS.

Running the SCONE CLI

Running the SCONE CLI container:

docker run -it --rm \
  --device /dev/isgx \

Alternatively, if you want to run a SCONE CLI inside the Kubernetes cluster, use the following commands:

cat > cli-pod.yaml <<EOF
apiVersion: v1
kind: Pod
  name: scone-cli
    app: scone-cli
  - name: sconeapps
    - name: cli
        - sh
        - -c
        - sleep 20000
kubectl create -f cli-pod.yaml
kubectl exec -it scone-cli -- sh

Provisioning CAS

To claim ownership over a CAS and to be able to register backups for it, one must perform the provisioning operation through the SCONE CLI. To do so, the owner needs the provisioning token and the CAS key hash, both provided by CAS after startup, and a CAS owner configuration file, a TOML file with a CAS configuration.

The example below shows an owner configuration file for CAS.

common_name = "mycas"
alt_names = ["mycas", "cas", "localhost"]

spid = "<spid>"
linkable_quotes = true
sp_key = "<sp_key>"
base_uri = ""

subscription_key = "<subscription key>"

To provision the primary CAS, run:

scone cas provision $SCONE_CAS_ADDR \
  -c $CAS_KEY_HASH \
  --config-file cas-default-owner-config.toml \
You need to export the appropriate values for

  • $SCONE_CAS_ADDR: primary CAS address; remember to use the same address as the one the CAS Backup Controller will contact!
  • $PROVISIONING_TOKEN: available at CAS logs after startup, and
  • $CAS_KEY_HASH: you can also find this in the CAS log.

The subcommand with-attestation attests and verifies the primary CAS and saves the owner identity to the SCONE CLI configuration file. In this example, the SCONE CLI configuration is saved at its default location, ~/.cas/config. To keep it somewhere else, set the environment variable SCONE_CLI_CONFIG to the desired path. You can pass optional arguments to relax the attestation constraints. For example, to allow older, less secure SGX hardware (-G) with hyperthreading enabled (-C) and enclaves in debug mode, use with-attestation -G -C --only_for_testing-debug.

The parameter --config-file points to the CAS owner configuration file.

After successful provisioning, the owner can register backups for the primary CAS. The owner identity is saved to the SCONE CLI configuration file.

Running the Controller

Running the CAS Backup Controller as a Kubernetes pod

Suppose the CAS Backup Controller is started on the Kubernetes cluster. In that case, the SCONE CLI configuration file must be protected since we no longer trust the CAS Backup Controller infrastructure. To run it securely, we must leverage SCONE's remote attestation and secure file injection mechanisms to securely deliver the SCONE CLI configuration file (which contains the owner identity) to a CAS Backup Controller running inside of SGX enclaves.

To do so, we must first create a policy for the Controller. For development, this looks like this:

name: backup_controller
version: "0.3"

    tolerate: [debug-mode, hyperthreading, outdated-tcb, software-hardening-needed]
    ignore_advisories: "*"

  - name: register
    image_name: cli
    mrenclaves: [307a470e9bc42e28fee9a5190c21821d8a2b2bbfb67ee175852e1635473c75b9]
    command: "scone cas register-backup @@3"
      SCONE_MODE: hw
      SCONE_LOG: error
      SCONE_CLI_CONFIG: /etc/owner-config.json
      \@\@SCONE_LAS_ADDR: ""
    pwd: /

  - name: cli
      - path: /etc/owner-config.json
        content: |
            "default_cas": "mycas",
            "identity": "-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgHyupqvcKNLNDz7Hz\n85NyYXmYKnDYvS9X+ajztepQexKhRANCAATKxsvoQtR4RGscgtzPOgTdYZuODe0W\nxlr74FcTfPOtmW8/2l3Uop73CWGoZ21iVIxJCo5ajDP0mgWGR73ebuCj\n-----END PRIVATE KEY-----\n-----BEGIN CERTIFICATE-----\nMIIBSzCB8aADAgECAgEqMAoGCCqGSM49BAMCMCExHzAdBgNVBAMMFnJjZ2VuIHNl\nbGYgc2lnbmVkIGNlcnQwIBcNNzUwMTAxMDAwMDAwWhgPNDA5NjAxMDEwMDAwMDBa\nMCExHzAdBgNVBAMMFnJjZ2VuIHNlbGYgc2lnbmVkIGNlcnQwWTATBgcqhkjOPQIB\nBggqhkjOPQMBBwNCAATKxsvoQtR4RGscgtzPOgTdYZuODe0Wxlr74FcTfPOtmW8/\n2l3Uop73CWGoZ21iVIxJCo5ajDP0mgWGR73ebuCjoxgwFjAUBgNVHREEDTALgglz\nY29uZSBjbGkwCgYIKoZIzj0EAwIDSQAwRgIhAOsHv3TLIm9zjvXwYXevqRU3kWW5\nWFfchJ7ZIo4Yu2wBAiEAhdH9Esc3f1ionob4s9n1PyyRy8p7hVpk7Wo4BaidAKs=\n-----END CERTIFICATE-----\n",
            "cas_db": {
              "mycas": {
                "url": "https://mycas:8081/",
                "ias_report": {
                  "request_id": "37339931456b431bbb223b8b42a0021f",
                  "report_signature": "ZwilghAYM4ZdiETPwoGIThBbwaePDuRe6OBEr8qUDHgi321IVcuBtY8mEvAgDlvvwHGB0YpzdR3zy7W1h07gTa76CrGEbX4IdkHZPFSphscapduYzpA0rYH796Iu4VVQOqKsd8L8rQ1iFmc2tSXkOvf6eic7mpzhkEkHGiPx21kY29/Bhby/+PCg8RKMWJktcOcN6b0NRugDALa7A0Hn5mK93lpXhaxe9Rycgi2HESx9gI23JNa5+RoJ3GRgtkTYAqR5Ay6XqjyEw4smjR/H2EJnrZGmJHRL/4oXDjpyLEmqpFO/IEypSdRaRi7LmJln0lpKOPO6YSw6sGAs3RZwgg==",
                  "signing_certificate": "-----BEGIN CERTIFICATE-----\nMIIEoTCCAwmgAwIBAgIJANEHdl0yo7CWMA0GCSqGSIb3DQEBCwUAMH4xCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTEUMBIGA1UEBwwLU2FudGEgQ2xhcmExGjAYBgNV\nBAoMEUludGVsIENvcnBvcmF0aW9uMTAwLgYDVQQDDCdJbnRlbCBTR1ggQXR0ZXN0\nYXRpb24gUmVwb3J0IFNpZ25pbmcgQ0EwHhcNMTYxMTIyMDkzNjU4WhcNMjYxMTIw\nMDkzNjU4WjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExFDASBgNVBAcMC1Nh\nbnRhIENsYXJhMRowGAYDVQQKDBFJbnRlbCBDb3Jwb3JhdGlvbjEtMCsGA1UEAwwk\nSW50ZWwgU0dYIEF0dGVzdGF0aW9uIFJlcG9ydCBTaWduaW5nMIIBIjANBgkqhkiG\n9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqXot4OZuphR8nudFrAFiaGxxkgma/Es/BA+t\nbeCTUR106AL1ENcWA4FX3K+E9BBL0/7X5rj5nIgX/R/1ubhkKWw9gfqPG3KeAtId\ncv/uTO1yXv50vqaPvE1CRChvzdS/ZEBqQ5oVvLTPZ3VEicQjlytKgN9cLnxbwtuv\nLUK7eyRPfJW/ksddOzP8VBBniolYnRCD2jrMRZ8nBM2ZWYwnXnwYeOAHV+W9tOhA\nImwRwKF/95yAsVwd21ryHMJBcGH70qLagZ7Ttyt++qO/6+KAXJuKwZqjRlEtSEz8\ngZQeFfVYgcwSfo96oSMAzVr7V0L6HSDLRnpb6xxmbPdqNol4tQIDAQABo4GkMIGh\nMB8GA1UdIwQYMBaAFHhDe3amfrzQr35CN+s1fDuHAVE8MA4GA1UdDwEB/wQEAwIG\nwDAMBgNVHRMBAf8EAjAAMGAGA1UdHwRZMFcwVaBToFGGT2h0dHA6Ly90cnVzdGVk\nc2VydmljZXMuaW50ZWwuY29tL2NvbnRlbnQvQ1JML1NHWC9BdHRlc3RhdGlvblJl\ncG9ydFNpZ25pbmdDQS5jcmwwDQYJKoZIhvcNAQELBQADggGBAGcIthtcK9IVRz4r\nRq+ZKE+7k50/OxUsmW8aavOzKb0iCx07YQ9rzi5nU73tME2yGRLzhSViFs/LpFa9\nlpQL6JL1aQwmDR74TxYGBAIi5f4I5TJoCCEqRHz91kpG6Uvyn2tLmnIdJbPE4vYv\nWLrtXXfFBSSPD4Afn7+3/XUggAlc7oCTizOfbbtOFlYA4g5KcYgS1J2ZAeMQqbUd\nZseZCcaZZZn65tdqee8UXZlDvx0+NdO0LR+5pFy+juM0wWbu59MvzcmTXbjsi7HY\n6zd53Yq5K244fwFHRQ8eOB0IWB+4PfM7FeAApZvlfqlKOlLcZL2uyVmzRkyR5yW7\n2uo9mehX44CiPJ2fse9Y6eQtcfEhMPkmHXI01sN+KwPbpA39+xOsStjhP9N1Y1a2\ntQAVo+yVgLgV2Hws73Fc0o3wC78qPEA+v2aRs/Be3ZFDgDyghc/1fgU+7C+P6kbq\nd4poyb6IW8KCJbxfMJvkordNOgOUUxndPHEi/tb/U7uLjLOgPA==\n-----END CERTIFICATE-----\n-----BEGIN CERTIFICATE-----\nMIIFSzCCA7OgAwIBAgIJANEHdl0yo7CUMA0GCSqGSIb3DQEBCwUAMH4xCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTEUMBIGA1UEBwwLU2FudGEgQ2xhcmExGjAYBgNV\nBAoMEUludGVsIENvcnBvcmF0aW9uMTAwLgYDVQQDDCdJbnRlbCBTR1ggQXR0ZXN0\nYXRpb24gUmVwb3J0IFNpZ25pbmcgQ0EwIBcNMTYxMTE0MTUzNzMxWhgPMjA0OTEy\nMzEyMzU5NTlaMH4xCzAJBgNVBAYTAlVTMQswCQYDVQQIDAJDQTEUMBIGA1UEBwwL\nU2FudGEgQ2xhcmExGjAYBgNVBAoMEUludGVsIENvcnBvcmF0aW9uMTAwLgYDVQQD\nDCdJbnRlbCBTR1ggQXR0ZXN0YXRpb24gUmVwb3J0IFNpZ25pbmcgQ0EwggGiMA0G\nCSqGSIb3DQEBAQUAA4IBjwAwggGKAoIBgQCfPGR+tXc8u1EtJzLA10Feu1Wg+p7e\nLmSRmeaCHbkQ1TF3Nwl3RmpqXkeGzNLd69QUnWovYyVSndEMyYc3sHecGgfinEeh\nrgBJSEdsSJ9FpaFdesjsxqzGRa20PYdnnfWcCTvFoulpbFR4VBuXnnVLVzkUvlXT\nL/TAnd8nIZk0zZkFJ7P5LtePvykkar7LcSQO85wtcQe0R1Raf/sQ6wYKaKmFgCGe\nNpEJUmg4ktal4qgIAxk+QHUxQE42sxViN5mqglB0QJdUot/o9a/V/mMeH8KvOAiQ\nbyinkNndn+Bgk5sSV5DFgF0DffVqmVMblt5p3jPtImzBIH0QQrXJq39AT8cRwP5H\nafuVeLHcDsRp6hol4P+ZFIhu8mmbI1u0hH3W/0C2BuYXB5PC+5izFFh/nP0lc2Lf\n6rELO9LZdnOhpL1ExFOq9H/B8tPQ84T3Sgb4nAifDabNt/zu6MmCGo5U8lwEFtGM\nRoOaX4AS+909x00lYnmtwsDVWv9vBiJCXRsCAwEAAaOByTCBxjBgBgNVHR8EWTBX\nMFWgU6BRhk9odHRwOi8vdHJ1c3RlZHNlcnZpY2VzLmludGVsLmNvbS9jb250ZW50\nL0NSTC9TR1gvQXR0ZXN0YXRpb25SZXBvcnRTaWduaW5nQ0EuY3JsMB0GA1UdDgQW\nBBR4Q3t2pn680K9+QjfrNXw7hwFRPDAfBgNVHSMEGDAWgBR4Q3t2pn680K9+Qjfr\nNXw7hwFRPDAOBgNVHQ8BAf8EBAMCAQYwEgYDVR0TAQH/BAgwBgEB/wIBADANBgkq\nhkiG9w0BAQsFAAOCAYEAeF8tYMXICvQqeXYQITkV2oLJsp6J4JAqJabHWxYJHGir\nIEqucRiJSSx+HjIJEUVaj8E0QjEud6Y5lNmXlcjqRXaCPOqK0eGRz6hi+ripMtPZ\nsFNaBwLQVV905SDjAzDzNIDnrcnXyB4gcDFCvwDFKKgLRjOB/WAqgscDUoGq5ZVi\nzLUzTqiQPmULAQaB9c6Oti6snEFJiCQ67JLyW/E83/frzCmO5Ru6WjU4tmsmy8Ra\nUd4APK0wZTGtfPXU7w+IBdG5Ez0kE1qzxGQaL4gINJ1zMyleDnbuS8UicjJijvqA\n152Sq049ESDz+1rRGc2NVEqh1KaGXmtXvqxXcTB+Ljy5Bw2ke0v8iGngFBPqCTVB\n3op5KBG3RjbF6RRSzwzuWfL7QErNC8WEy5yDVARzTA5+xmBc388v9Dm21HGfcC8O\nDD+gT9sSpssq0ascmvH49MOgjt1yoysLtdCtJW/9FZpoOypaHx0R+mJTLwPXVMrv\nDaVzWh5aiEx+idkSGMnX\n-----END CERTIFICATE-----\n",
                  "advisory_url": null,
                  "advisory_ids": null,

Update the content field of the injection file being rendered at /etc/owner-config.json with the contents of the SCONE CLI configuration file after the provisioning process. This way, the CAS owner's identity is protected in the enclave.

After the session file is updated, please submit it to the primary CAS.

scone session create session.yml

To deploy the CAS Backup Controller, use its Helm charts from the SconeAppsEE repository. Note that we are setting session to be the same as the SCONE_CONFIG_ID of the session we have just created (backup_controller/register).

helm install cas-backup-controller sconeappsee/cas-backup-controller \
   --set session=backup_controller/register \
   --set cas=mycas \
   --set backupName=mycas-backup

For an end-to-end example, check the Quick Start guide.

Environment variables

You can use the following environment variables to customize the Controller execution.

  • BACKUP_CAS_IMAGE is the container image used to deploy the ephemeral backup CAS. The default value is

  • CONTROLLER_LOOP_INTERVAL_SECONDS how often the Controller should check for nodes and try to register backups. The default value is 60.

  • BACKUP_CAS_NAME is the base name of the backup CAS resources created by the Controller on the Kubernetes cluster. If not set, the Controller will generate a valid base name from the primary CAS address.

  • NAMESPACE is the Kubernetes namespace to which the Controller will deploy backup resources. The default value is default.

  • SCONE_LAS_ADDR is the address of LAS, if remote attestation is enabled. The default value is (the default Docker network interface).

  • SGX_RESOURCE_NAME defines the Kubernetes resource name for SGX devices. The default value is

  • SGX_RESOURCE_QUANTITY defines the quantity of the Kubernetes resources for SGX devices to be requested. The default value is 1.

  • SELECTOR_LABEL defines a label selector for nodes. Useful for heterogeneous clusters. The default value is None (i.e., do not filter nodes).

  • LAS_SELECTOR_LABEL defines a label selector for LAS pods. This allows the Controller to use LAS public keys to track hardware changes. Default value is "" (i.e., do not look for LAS pods).

  • LAS_NAMESPACE defines the namespace where the Controller should look for LAS pods. The default value is default.

  • SCONE_CLI_CONFIG defines the fallback SCONE CLI configuration if none is provided through the --config parameter.