Skip to content

A Confidential Flask-Based Application

We demonstrate with the help of a simple Flask-based Service multiple features of the SCONE platform:

  • we show that we can execute unmodified Python programs inside of SGX enclaves
  • we show how to encrypt the Python program to protect the confidentiality and integrity of the Python code
  • how to implicitly attest other services with the help of TLS, i.e., to ensure that one communicates with a service that satisfy its security policy.
    • we demonstrate how Redis, an in-memory data structure store, and the Python flask attest each other via TLS without needing to change the code of neither Redis nor the Flask-based service.
  • we show how to generate TLS certificates with the help of a policy:
    • a SCONE security policy describes how to attest applications and services (i.e., describe the code, the filesystem state, the environment, the node on which to execute, and secrets).
    • a SCONE policy can generate secrets and in particular, key-pairs and TLS certificates.
  • we show how to execute this example
    • on a local computer with the help of docker-compose
    • on a generic Kubernetes cluster, and
    • on Azure Kubernetes Service (AKS).

Mutual Protection

Next Step

In the second version of this example, we simplify the workflow in the sense that we use a generic script to transform an existing native container image into an encrypted, confidential container image.

Flask-Based Confidential Service

We implement a simple Flask-based service. The Python code implements a REST API:

  • to store patient records (i.e., POST to resource /patient/<string:patient_id>)
  • to retrieve patient records (i.e., GET of resource /patient/<string:patient_id>)
  • to retrieve some score for a patient (i.e., GET of ressource '/score/<string:patient_id>)

The Python code is executed inside of an enclave to ensure that even users with root access cannot read the patient data.

TLS Certificates

The service uses a Redis instance to store the resources. The communication between 1) the Flask-based service and its clients and 2) Redis and the application is encrypted with the help of TLS. To do so, we need to provision the application and Redis with multiple keys and certificates:

  • Redis client certificate
  • Redis server certificate
  • Flask server certificate

Redis and the Flask-based service, require that the private keys and certificates are stored in the filesystem. We generate and provision these TLS-related files with the help of a SCONE policy.

To do so, we generate secrets related to the Flask-based service. We specify in the flask policy that

  • a private key (api_ca_key) for a new certificate authority (CA) is generated
  • a certificate (api_ca_cert) for a certification authority is generated
  • using the private key (i.e., api_ca_key), and
  • making this certificate available to everybody (see export_public: true)
  • we generate a private key for the certificate used by the REST API (i.e., flask_key)
  • we generate a certificate (flask) with the help of CA api_ca_cert and assign it a dns name api.

The SCONE policy is based on Yaml and the flask policy contains the following section to define these secrets:

secrets:
    - name: api_ca_key
      kind: private-key
    - name: api_ca_cert
      kind: x509-ca
      export_public: true
      private_key: api_ca_key
    - name: flask_key
      kind: private-key
    - name: flask
      kind: x509
      private_key: flask_key
      issuer: api_ca_cert
      dns:
        - api

The private keys and certificates are expected at certain locations in the file system. SCONE permits to map these secrets into the filesystem of the Flask-based service: these files are only visible to the service inside of an SGX enclave after a successful attestation (see below) and in particular, not visible on the outside i.e., in the filesystem of the container.

To map the private keys and certificates into the filesystem of a service, we specify in the policy which secrets are visible to a service at which path. In the flask policy this is done as follows:

images:
   - name: flask_restapi_image
     injection_files:
        - path: /tls/flask.crt
          content: $$SCONE::flask.crt$$
        - path: /tls/flask.key
          content: $$SCONE::flask.key$$

And in the Python program, one can just access these files as normal files. One can create a SSL context (see code):

    app.run(host='0.0.0.0', port=4996, threaded=True, ssl_context=(("/tls/flask.crt", "/tls/flask.key")))

While we do not show how to enforce client authentication of the REST API, we show how to do this for Redis in the next section.

TLS-based Mutual Attestation

The communication between the Flask-based service, say, S and Redis instance, say, R is encrypted via TLS. Actually, we make sure that the service S and instance R attest each other. Attestation means that S ensures that R satisfies all requirements specified in R's security policy and R ensures that S satisfies all the requirements of S's policy. Of course, this should be done without changing the code of neither S nor R. In case that S and R are using TLS with client authentication, this is straightforward to enforce. (If this is not the case, please contact us for an alternative.)

To ensure mutual attestation, the operator of Redis defines a policy in which it defines a certification authority (redis_ca_cert) and defines both a Redis certificate (redis_ca_cert) as well as a Redis client certificate (redis_client_cert). The client certificate and the private key (redis_client_key) are exported to the policy of the Flask service S. The policy for this looks like this:

secrets:
  - name: redis_key
    kind: private-key
  - name: redis # automatically generate Redis server certificate
    kind: x509
    private_key: redis_key
    issuer: redis_ca_cert
    dns:
     - redis
  - name: redis_client_key
    kind: private-key
    export:
    - session: $FLASK_SESSION
  - name: redis_client_cert # automatically generate client certificate
    kind: x509
    issuer: redis_ca_cert
    private_key: redis_client_key
    export:
    - session: $FLASK_SESSION # export client cert/key to client session
  - name: redis_ca_key
    kind: private-key
  - name: redis_ca_cert # export session CA certificate as Redis CA certificate
    kind: x509-ca
    private_key: redis_ca_key
    export:
    - session: $FLASK_SESSION # export the session CA certificate to client session

Note that $FLASK_SESSION is replaced by the unique name of the policy of S. The security policies are in this example on the same SCONE CAS (Configuration and Attestation Service). In more complex scenarios, the policies could also be stored on separate SCONE CAS instances operated by different entities.

The flask service can import the Redis CA certificate, client certificate and private key as follows:

secrets:
    - name: redis_client_key
      import:
        session: $REDIS_SESSION
        secret: redis_client_key
    - name: redis_client_cert
      import:
        session: $REDIS_SESSION
        secret: redis_client_cert
    - name: redis_ca_cert
      import:
        session: $REDIS_SESSION
        secret: redis_ca_cert

These secrets are made available to the Flask-based service in the filesystem (i.e., files /tls/redis-ca.crt, /tls/client.crt and /tls/client.key) via the following entries in its security policy:

images:
   - name: flask_restapi_image
     injection_files:
        - path: /tls/redis-ca.crt
          content: $$SCONE::redis_ca_cert.chain$$
        - path: /tls/client.crt
          content: $$SCONE::redis_client_cert.crt$$
        - path: /tls/client.key
          content: $$SCONE::redis_client_cert.key$$

Note that before uploading a policy to SCONE CAS, one first attests that one indeed communicates with a genuine SCONE CAS running inside of a production enclave. This is done with the help of a SCONE CAS CLI.

Code

The source code of this example is open source and available on github:

git clone https://github.com/scontain/flask_example.git
cd flask_example

Run Service On Local Computer

You can use docker-compose to run this example on your local SGX-enabled computer as follows. You first generate an encrypted image using script create_image.sh. This generates some environment variables that stored in file myenv and are loaded via source myenv. The service and Redis are started with docker-compose up.

./create_image.sh
source myenv
docker-compose up

We use a public instance of SCONE CAS in this example.

Testing the service

Retrieve the API certificate from CAS:

source myenv
curl -k -X GET "https://${SCONE_CAS_ADDR-cas}:8081/v1/values/session=$FLASK_SESSION" | jq -r .values.api_ca_cert.value > cacert.pem

Since the API certificates are issued to the host name "api", we have to use it. You can rely on cURL's --resolve option to point to the actual address (you can also edit your /etc/hosts file).

export URL=https://api:4996
curl --cacert cacert.pem -X POST ${URL}/patient/patient_3 -d "fname=Jane&lname=Doe&address='123 Main Street'&city=Richmond&state=Washington&ssn=123-223-2345&email=nr@aaa.com&dob=01/01/2010&contactphone=123-234-3456&drugallergies='Sulpha, Penicillin, Tree Nut'&preexistingconditions='diabetes, hypertension, asthma'&dateadmitted=01/05/2010&insurancedetails='Primera Blue Cross'" --resolve api:4996:127.0.0.1
curl --cacert cacert.pem -X GET ${URL}/patient/patient_3 --resolve api:4996:127.0.0.1
curl --cacert cacert.pem -X GET ${URL}/score/patient_3 --resolve api:4996:127.0.0.1

The output might look as follows:

$ curl --cacert cacert.pem -X POST https://localhost:4996/patient/patient_3 -d "fname=Jane&lname=Doe&address='123 Main Street'&city=Richmond&state=Washington&ssn=123-223-2345&email=nr@aaa.com&dob=01/01/2010&contactphone=123-234-3456&drugallergies='Sulpha, Penicillin, Tree Nut'&preexistingconditions='diabetes, hypertension, asthma'&dateadmitted=01/05/2010&insurancedetails='Primera Blue Cross'" --resolve api:4996:127.0.0.1
{"address":"'123 Main Street'","city":"Richmond","contactphone":"123-234-3456","dateadmitted":"01/05/2010","dob":"01/01/2010","drugallergies":"'Sulpha, Penicillin, Tree Nut'","email":"nr@aaa.com","fname":"Jane","id":"patient_3","insurancedetails":"'Primera Blue Cross'","lname":"Doe","preexistingconditions":"'diabetes, hypertension, asthma'","score":0.1168424489618366,"ssn":"123-223-2345","state":"Washington"}
$ curl --cacert cacert.pem -X GET localhost:4996/patient/patient_3 --resolve api:4996:127.0.0.1
{"address":"'123 Main Street'","city":"Richmond","contactphone":"123-234-3456","dateadmitted":"01/05/2010","dob":"01/01/2010","drugallergies":"'Sulpha, Penicillin, Tree Nut'","email":"nr@aaa.com","fname":"Jane","id":"patient_3","insurancedetails":"'Primera Blue Cross'","lname":"Doe","preexistingconditions":"'diabetes, hypertension, asthma'","score":0.1168424489618366,"ssn":"123-223-2345","state":"Washington"}
$ curl --cacert cacert.pem -X GET localhost:4996/score/patient_3 --resolve api:4996:127.0.0.1
{"id":"patient_3","score":0.2781606437899131}

Execution on a Kubernetes Cluster and AKS

You can run this example on a Kubernetes cluster or Azure Kubernetes Service (AKS).

Install SCONE services

Get access to SconeApps (see https://sconedocs.github.io/helm/):

helm repo add sconeapps https://${GH_TOKEN}@raw.githubusercontent.com/scontain/sconeapps/master/
helm repo update

Give SconeApps access to the private docker images by generating an access token on https://gitlab.scontain.com/-/profile/personal_access_tokens to read_registry and set:

export SCONE_HUB_USERNAME=...
export SCONE_HUB_ACCESS_TOKEN=...
export SCONE_HUB_EMAIL=...

kubectl create secret docker-registry sconeapps --docker-server=registry.scontain.com --docker-username=$SCONE_HUB_USERNAME --docker-password=$SCONE_HUB_ACCESS_TOKEN --docker-email=$SCONE_HUB_EMAIL

Start LAS:

helm install las sconeapps/las  --set image=registry.scontain.com/sconecuratedimages/kubernetes:las-scone4.2.1 --set service.hostPort=true

If you use a local cas, you can start this cas service as follows:

helm install cas sconeapps/cas --set image=registry.scontain.com/sconecuratedimages/services:cas-scone4.2.1

Install the SGX device plugin for Kubernetes:

helm install sgxdevplugin sconeapps/sgxdevplugin

Run the application

Start by creating a Docker image and setting its name. Remember to specify a repository to which you are allowed to push:

export IMAGE=registry.scontain.com/sconecuratedimages/application:v0.4 # please change to an image that you can push
./create_image.sh
source myenv
docker push $IMAGE

Use the Helm chart in deploy/helm to deploy the application to a Kubernetes cluster.

helm install api-v1 deploy/helm \
   --set image=$IMAGE \
   --set scone.cas=$SCONE_CAS_ADDR \
   --set scone.flask_session=$FLASK_SESSION/flask_restapi \
   --set scone.redis_session=$REDIS_SESSION/redis \
   --set service.type=NodePort \
   --set redis.image=registry.scontain.com/sconecuratedimages/apps:redis-6-alpine-scone4.2.1

NOTE: Setting service.type=LoadBalancer will allow the application to get traffic from the internet (through a managed LoadBalancer).

Test the application

After all resources are Running, you can test the API via Helm:

helm test api-v1

Helm will run a pod with a couple of pre-set queries to check if the API is working properly.

Access the application

If the application is exposed to the world through a service of type LoadBalancer, you can retrieve its CA certificate from CAS:

source myenv
curl -k -X GET "https://${SCONE_CAS_ADDR-cas}:8081/v1/values/session=$FLASK_SESSION" | jq -r .values.api_ca_cert.value > cacert.pem

Retrieve the service public IP address:

export SERVICE_IP=$(kubectl get svc --namespace default api-v1-example --template "{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}")

Since the API certificates are issued to the host name "api", we have to use it. You can rely on cURL's --resolve option to point to the actual address (you can also edit your /etc/hosts file).

export URL=https://api

Now you can perform queries such as:

curl --cacert cacert.pem -X POST ${URL}/patient/patient_3 -d "fname=Jane&lname=Doe&address='123 Main Street'&city=Richmond&state=Washington&ssn=123-223-2345&email=nr@aaa.com&dob=01/01/2010&contactphone=123-234-3456&drugallergies='Sulpha, Penicillin, Tree Nut'&preexistingconditions='diabetes, hypertension, asthma'&dateadmitted=01/05/2010&insurancedetails='Primera Blue Cross'" --resolve api:443:${SERVICE_IP}

Clean up

helm delete cas
helm delete las
helm delete sgxdevplugin
helm delete api-v1
kubectl delete pod api-v1-example-test-api

Next, we introduce the different sconified Python versions that we support.