SCONE Online Boutique example
This demo showcases how the sconify-image tool can be used to turn existing microservices into confidential containers, with encrypted filesystems and remote attestation. For that purpose, we convert the Google Online Boutique demo, a well-known microservices application composed by 10 services in different languages, to run on SCONE.
Demo
Requirements
This demo is executed in two stages:
Stage 1: One-step image transformation (sconify)
On a trusted machine, sconify-image will transform all standard images into confidential images that run on top of SCONE, create security policies for remote attestation and generate Kubernetes manifests or Helm charts. Check the sconify documentation to learn more about this process.
Requirements:
- Docker.
sconify-image.
Stage 2: Deployment
Using the generated deployment resources from the previous step, deploy the confidential services to a Kubernetes cluster with Intel SGX support. You don't need to trust or control the cluster nodes or the deployment process.
Requirements:
- Kubernetes cluster with Intel SGX support.
- SCONE LAS running on the cluster. Learn more.
TL;DR
git clone https://github.com/scontain/microservices-demo -b sconify
cd microservices-demo
# Specify the registry to which the confidential images will be pushed.
# You must be logged in and have push permission.
export IMAGE_REPOSITORY="docker.io/myrepo/online-boutique"
# Configure Kubernetes namespace (required here so that the in-cluster
# DNS names of the microservices are part of the security policies).
export K8S_NAMESPACE="default"
./sconify-all.sh
# Configure an imagePullSecret for Kubernetes, if necessary.
export K8S_IMAGE_PULL_SECRET="sconeapps"
# Configure which SGX device plugin you want to use ("scone", "azure" or "disabled")
export K8S_DEVICE_PLUGIN="azure"
./deploy-all.sh
One-step image transformation
SCONE is a lift-and-shift framework for confidential containers. With that in mind, we support a one-step transformation process for standard containers (i.e., not confidential) into confidential containers. We call the process sconify (learn more). This means that existing code and workflows (e.g., CI/CD pipelines) can benefit from SCONE guarantees with little-to-no effort.
Go services support
SCONE requires Go services to be built with gccgo (learn more). For this reason, we use alternative Dockerfiles to build the standard services (gcc.Dockerfile). Note that the original source code is untouched.
C# services support
SCONE support for C#. For this reason, sconify-image does not support C# yet, and we provide a pre-built image for the C# service of the demo, cartservice, with SCONE. Check src/cartservice/sconify.sh and policies/cartservice.yaml for more information.
Compared to the original repository, the following directories are modified. Note that no source code changes are performed.
policies/ # Code to create a CAS namespace and creating policies for services that are not sconified (redis, cartservice)
release/charts/ # Helm charts for services that not sconified (redis, cartservice). Generated Helm charts are saved here too
src/ # Original source code, sconify.sh scripts and alternative Dockerfiles (if necessary)
In this example, we have an online boutique composed of the following services (the description was taken from the original repository):
| Service | Language | Description |
|---|---|---|
| frontend | Go | Exposes an HTTP server to serve the website. Does not require signup/login and generates session IDs for all users automatically. |
| cartservice | C# | Stores the items in the user's shopping cart in Redis and retrieves it. |
| productcatalogservice | Go | Provides the list of products from a JSON file and ability to search products and get individual products. |
| currencyservice | Node.js | Converts one money amount to another currency. Uses real values fetched from European Central Bank. It's the highest QPS service. |
| paymentservice | Node.js | Charges the given credit card info (mock) with the given amount and returns a transaction ID. |
| shippingservice | Go | Gives shipping cost estimates based on the shopping cart. Ships items to the given address (mock) |
| emailservice | Python | Sends users an order confirmation email (mock). |
| checkoutservice | Go | Retrieves user cart, prepares order and orchestrates the payment, shipping and the email notification. |
| recommendationservice | Python | Recommends other products based on what's given in the cart. |
| adservice | Java | Provides text ads based on given context words. |
We perform the one-step transformation of all the services above. Each service has its own sconify.sh script with the transformation parameters (e.g., src/emailservice/sconify.sh). sconify-image ships with sensible defaults so it's easy to get started, but it also supports a fair amount of configuration parameters, should you require more customization or additional integrations (e.g., integration to Azure services).
The flow is as follows:
- Build standard image (using the original source code).
- Transform it into a confidential image. Push the generated image to a container registry.
- Generate a security policy. Push it to a SCONE CAS.
- Generate Helm charts (or Kubernetes manifests).
You can trigger the sconification of a single service by running their own sconify.sh. To transform all services at once, run sconify-all.sh, which simply loops through all services and calls their sconify.sh script.
# Specify the registry to which the confidential images will be pushed.
# You must be logged in and have push permission.
export IMAGE_REPOSITORY="docker.io/myrepo/online-boutique"
# Configure Kubernetes namespace (required here so that the in-cluster
# DNS names of the microservices are part of the security policies).
export K8S_NAMESPACE="default"
./sconify-all.sh
This will transform all images, and now the generated Helm charts will be in release/charts.
sconify-all.sh supports the following environment variables:
| Environment variable | Description | Default value |
|---|---|---|
| IMAGE_REPOSITORY | Container image repository for the generated confidential images | |
| SCONE_CAS_ADDR | Address of SCONE CAS that will receive the generated security policies | "5-4-0.scone-cas.cf" |
| CAS_MRENCLAVE | The MrEnclave of the chosen SCONE CAS (for attestation and trust establishment) | "9f82c11af7ed3c3212483a5ad33b59923ca1462d1814943ff2586932c21fa51b" |
| K8S_NAMESPACE | The Kubernetes namespace for deploying the services, since the microservices talk to each other using their in-cluster DNS names | "default" |
This example also uses redis as the storage backend for cartservice. There is no need to sconify redis, as we already provide it as one of our curated images.
The following service is part of the demo but does not run on SCONE (as it is just a benchmark):
| Service | Language | Description |
|---|---|---|
| loadgenerator | Python/Locust | Continuously sends requests imitating realistic user shopping flows to the frontend. |
Deployment
After the Helm charts are generated, one can deploy them to a Kubernetes cluster. Similar to sconify-all.sh, we provide a deploy-all.sh script, which loops through all Helm charts in release/charts and install them. The following environment variables are accepted by deploy-all.sh:
| Environment variable | Description | Default value |
|---|---|---|
| K8S_IMAGE_PULL_SECRET | You need a Secret to access private container registries. Learn more | "sconeapps" |
| K8S_DEVICE_PLUGIN | Configure which device plugin to use: "scone" for SCONE SGX device plugin, "azure" for the Azure SGX device plugin or "disabled" to access the SGX devices via hostPath volumes | "scone" |
| K8S_FRONTEND_SERVICE_TYPE | Configure Service type of "frontend", i.e., how to expose it (LoadBalancer or NodePort) | "NodePort" |
To deploy the services to an AKS cluster, for example, run:
# Configure an imagePullSecret for Kubernetes, if necessary.
export K8S_IMAGE_PULL_SECRET="sconeapps"
# Configure which SGX device plugin you want to use ("scone", "azure" or "disabled")
export K8S_DEVICE_PLUGIN="azure"
# Configure Service type of "frontend", i.e., how to expose it (LoadBalancer or NodePort)
export K8S_FRONTEND_SERVICE_TYPE="LoadBalancer"
./deploy-all.sh
After all charts are deployed, you can see their pods running:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
adservice-sconify-adservice-5c5d79cd5-hmpmv 1/1 Running 0 7m44s
cartservice-sconify-cartservice-557767665f-79ltl 1/1 Running 0 7m39s
checkoutservice-sconify-checkoutservice-69cbcc9cf6-rsbpv 1/1 Running 0 7m33s
currencyservice-sconify-currencyservice-d7cb458db-vtwqq 1/1 Running 0 7m28s
emailservice-sconify-emailservice-84fb77d787-6gbhh 1/1 Running 0 7m22s
frontend-sconify-frontend-77fdf9d665-mb2wh 1/1 Running 0 6m54s
las-49fzx 1/1 Running 0 8h
las-gjgr8 1/1 Running 0 8h
paymentservice-sconify-paymentservice-79cd585dc9-nzqk5 1/1 Running 0 7m17s
productcatalogservice-sconify-productcatalogservice-75bfc9pxbtg 1/1 Running 0 7m11s
recommendationservice-sconify-recommendationservice-5c9bcbpghkm 1/1 Running 0 7m5s
redis-cart-69b6f44897-nn45s 1/1 Running 0 7m50s
shippingservice-sconify-shippingservice-68d544fcdc-v4h5g 1/1 Running 0 7m
You can now access the online boutique by fetching the frontend service address. If you set K8S_FRONTEND_SERVICE_TYPE to LoadBalancer, you should see the EXTERNAL-IP field populated for the frontendservice.