Skip to content

Running Java Applications in Scone with CAS-Policy

Requirements

The following steps assume that you have a running CAS container and a running LAS server and know its addresses.

Some of the latest containers for CAS and LAS are available at

docker pull registry.scontain.com/sconecuratedimages/services:cas
docker pull registry.scontain.com/sconecuratedimages/kubernetes:las

but the service repository requires a commercial subscription. You can use our public CAS instance at domain scone-cas.cf instead.

Make sure the sgx driver and docker is installed and running correctly by doing the java hello world sample. We determine which SGX device to mount with function determine_sgx_device.

determine_sgx_device
docker run $MOUNT_SGXDEVICE -it registry.scontain.com/sconecuratedimages/apps:openjdk-8-alpine

or, in case the container does not start the bash command automatically:

docker run $MOUNT_SGXDEVICE -it registry.scontain.com/sconecuratedimages/apps:openjdk-8-alpine /bin/sh

For Java 11 or Java 15 instead of Java 8:

docker run $MOUNT_SGXDEVICE -it registry.scontain.com/sconecuratedimages/apps:openjdk-11-alpine /bin/sh
docker run $MOUNT_SGXDEVICE -it registry.scontain.com/sconecuratedimages/apps:openjdk-15-alpine /bin/sh

We provide a quick hello-world sample in Java

cat > HelloWorld.java << EOF
import java.util.Map;
public class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hello World");
        Map<String, String> env = System.getenv();
        for (String envName : env.keySet()) {
            System.out.format("%s=%s%n", envName, env.get(envName));
        }
    }
}
EOF

Compile it, by executing:

javac HelloWorld.java

Expected output:

# javac HelloWorld.java
Picked up JAVA_TOOL_OPTIONS: -Xmx256m
#

You need to set some environment variables:

export SCONE_CAS_ADDR=scone-cas.cf  # we use the public CAS service
export SCONE_LAS_ADDR=127.0.0.1 # we must run a local LAS service
export SCONE_VERSION=1   # show the SCONE version
export SCONE_LOG=7   # maximum LOG level
java HelloWorld

Expected output after 2-3 minutes with a NUC and 8GB memory. Note that if you have a CPU that supports EDMM (Enclave dynamic memory management), the startup times will be much quicker.

/ # export SCONE_VERSION=1
/ # export SCONE_LOG=7
/ # java HelloWorld
export SCONE_QUEUES=4
export SCONE_SLOTS=256
export SCONE_SIGPIPE=0
export SCONE_MMAP32BIT=0
export SCONE_SSPINS=100
export SCONE_SSLEEP=4000
export SCONE_LOG=7
export SCONE_HEAP=4294967296
export SCONE_STACK=2097152
export SCONE_CONFIG=/etc/sgx-musl.conf
export SCONE_ESPINS=10000
export SCONE_MODE=hw
export SCONE_ALLOW_DLOPEN=yes (unprotected)
export SCONE_MPROTECT=yes
musl version: 1.1.24
Revision: efb2bdadba60c120f36864a1d675c4c4ca35ed69 (Tue Apr 28 06:29:26 2020 +0000)
Branch: 6ab648c20350b71cfeb8468001e8ebd78779870a

Enclave hash: 7954791612f147832807e1290604c11a6cefb6ff12fb97a6b51530be5004aecb
[SCONE|WARN] src/syscall/syscall.c:698:__scone_syscall_unshielded(): system call: SYS_membarrier, number 324 is not implemented.
[SCONE|WARN] src/syscall/syscall.c:698:__scone_syscall_unshielded(): system call: SYS_membarrier, number 324 is not implemented.
Picked up JAVA_TOOL_OPTIONS: -Xmx256m
[SCONE|WARN] src/shielding/proc_fs.c:368:_proc_fs_open(): open: /proc/self/mountinfo is not supported
[SCONE|WARN] src/shielding/proc_fs.c:368:_proc_fs_open(): open: /proc/self/maps is not supported
OpenJDK 64-Bit Server VM warning: Can't detect primordial thread stack location - find_vma failed
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
Hello World
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/jvm/java-1.8-openjdk/jre/bin:/usr/lib/jvm/java-1.8-openjdk/bin
SCONE_VERSION=1
SCONE_HEAP=4G
SCONE_LOG=7
SCONE_CAS_ADDR=scone-cas.cf
JAVA_HOME=/usr/lib/jvm/java-1.8-openjdk
TERM=xterm
LANG=C.UTF-8
SCONE_ALPINE=1
SCONE_ALLOW_DLOPEN=2
HOSTNAME=86fd2f848e41
SCONE_MPROTECT=1
SCONE_LAS_ADDR=127.0.0.1
LD_LIBRARY_PATH=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64
JAVA_TOOL_OPTIONS=-Xmx256m
PWD=/
HOME=/root
SHLVL=2
/ #

Posting a SCONE-Policy to configure the execution inside of Enclave

Using the CAS container to provision an execution inside of an enclave with environment variables etc. which were previously set. Therefore, the SCONE-Policy for Java executions must include the following parameters LD_LIBRARY_PATH and if necessary JAVA_TOOL_OPTIONS and the CLASSPATH of the required Java libraries:

First, similar to the posting sessions tutorial, we create a yaml file for java 1.8 openjdk

cat > sessionJavaHelloWorld.yml <<EOF
name: java
services:
   - name: hello
     mrenclaves: [7954791612f147832807e1290604c11a6cefb6ff12fb97a6b51530be5004aecb] # MRENCLAVE = Enclave hash: from execution with SCONE_VERSION=1
     command: "java HelloWorld"
     pwd: /
     environment:
       LD_LIBRARY_PATH: "/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64"
       JAVA_TOOL_OPTIONS: "-Xmx256m"
       CLASSPATH: "/" # path to directory contained your code
       TMP_SECRET_VAR: "This is a protected secret distributed by Scone CAS!"
EOF

We create client certificate and key material to identify this client: 

    mkdir -p conf
    if [[ ! -f conf/client.crt || ! -f conf/client-key.key  ]] ; then
        openssl req -x509 -newkey rsa:4096 -out conf/client.crt -keyout conf/client-key.key  -days 31 -nodes -sha256 -subj "/C=US/ST=Dresden/L=Saxony/O=Scontain/OU=Org/CN=www.scontain.com" -reqexts SAN -extensions SAN -config <(cat /etc/ssl/openssl.cnf \
    <(printf '[SAN]\nsubjectAltName=DNS:www.scontain.com'))
    fi

Next, we will upload this session to CAS:

curl -k -s --cert conf/client.crt --key conf/client-key.key --data-binary @sessionJavaHelloWorld.yml -X POST https://$SCONE_CAS_ADDR:8081/session

Expected output (please ignore the 'not-supported' fields):

Created Session[id=b11ac009c564f6e0c8c2b61d7b4774e89f896e83ec09ef3f919bd420aa3ab252, name='not-supported', status='not-supported']

If the output is not giving out an idea the session was not created. One possible reasons can be formatting errors like not using quotation marks ("java Helloworld") or single quotation marks ('java HelloWorld')

If we have set up the SCONE_CAS_ADDR and SCONE_LAS_ADDR, we export the environment variable according to the policy's name "java" and the service "hello"

export SCONE_CONFIG_ID="java/hello"
export SCONE_CAS_ADDR=$YOUR_CAS_ADDR
export SCONE_LAS_ADDR=$YOUR_LAS_ADDR
export SCONE_VERSION=1 
export SCONE_LOG=7

Now if we start the command "java HelloWorld" again, we see the following output.

/ # java HelloWorld
export SCONE_QUEUES=4
export SCONE_SLOTS=256
export SCONE_SIGPIPE=0
export SCONE_MMAP32BIT=0
export SCONE_SSPINS=100
export SCONE_SSLEEP=4000
export SCONE_LOG=7
export SCONE_HEAP=4294967296
export SCONE_STACK=2097152
export SCONE_CONFIG=/etc/sgx-musl.conf
export SCONE_ESPINS=10000
export SCONE_MODE=hw
export SCONE_ALLOW_DLOPEN=yes (unprotected)
export SCONE_MPROTECT=yes
musl version: 1.1.24
Revision: efb2bdadba60c120f36864a1d675c4c4ca35ed69 (Tue Apr 28 06:29:26 2020 +0000)
Branch: 6ab648c20350b71cfeb8468001e8ebd78779870a

Enclave hash: 7954791612f147832807e1290604c11a6cefb6ff12fb97a6b51530be5004aecb
[SCONE|INFO] src/shielding/crypto.c:222:crypto_get_identity(): Generated enclave certificate
[SCONE|INFO] src/shielding/eai_attestor.c:174:eai_attestor_init(): Created TLS context to communicate with CAS
[SCONE|DEBUG] src/shielding/eai_attestor.c:288:eai_attestor_attest(): Sending Attestation Request to LAS
[SCONE|DEBUG] src/shielding/eai_attestor.c:311:eai_attestor_attest(): Got Quote from LAS
[SCONE|DEBUG] src/shielding/eai_attestor.c:324:eai_attestor_attest(): Sending enclave hello message to CAS
[SCONE|DEBUG] src/shielding/eai_attestor.c:349:eai_attestor_attest(): Successfully attested enclave via CAS
[SCONE|DEBUG] src/process/init.c:639:__scone_prepare_secure_config(): Sending configuration request to CAS!
[SCONE|DEBUG] src/process/init.c:666:__scone_prepare_secure_config(): Received configuration from CAS!
[SCONE|WARN] src/syscall/syscall.c:698:__scone_syscall_unshielded(): system call: SYS_membarrier, number 324 is not implemented.
[SCONE|WARN] src/syscall/syscall.c:698:__scone_syscall_unshielded(): system call: SYS_membarrier, number 324 is not implemented.
Picked up JAVA_TOOL_OPTIONS: -Xmx256m
[SCONE|WARN] src/shielding/proc_fs.c:368:_proc_fs_open(): open: /proc/self/mountinfo is not supported
[SCONE|WARN] src/shielding/proc_fs.c:368:_proc_fs_open(): open: /proc/self/maps is not supported
OpenJDK 64-Bit Server VM warning: Can't detect primordial thread stack location - find_vma failed
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
[SCONE|WARN] src/syscall/anon.c:144:mmap_anon(): Protected heap memory exhausted! Set SCONE_HEAP environment variable to increase it.
Hello World
TMP_SECRET_VAR=This is a protected secret distributed by Scone CAS!
CLASSPATH=/
JAVA_TOOL_OPTIONS=-Xmx256m
LD_LIBRARY_PATH=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64
/ #

Note that the content of TMP_SECRET_VAR came from the Scone policy which was posted earlier to the CAS.

And we see the following logfiles in the containers of CAS (in case you are running a local copy):

ubuntu@kmaster:~$ docker logs cas_cas_1
[2020-05-08T11:32:02Z DEBUG rustls::server::hs] decided upon suite SupportedCipherSuite { suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, kx: ECDHE, bulk: AES_256_GCM, hash: SHA384, sign: ECDSA, enc_key_len: 32, fixed_iv_len: 4, explicit_nonce_len: 8, hkdf_algorithm: Algorithm(Algorithm(SHA384)) }
[2020-05-08T11:32:02Z DEBUG rustls::server::tls12] Session saved
[2020-05-08T11:32:02Z DEBUG eai::convert] Received length of next message
[2020-05-08T11:32:02Z DEBUG eai::convert] Next message will be 518 bytes long
[2020-05-08T11:32:02Z DEBUG eai::convert] Protobuf message received completely, interpreting it...
[2020-05-08T11:32:02Z DEBUG eai] Got message EnclaveMessage::EnclaveHello from enclave
[2020-05-08T11:32:02Z DEBUG cas::api::enclave::connection] V4(192.168.0.101:58584): In current state: Fresh got message: EnclaveMessage::EnclaveHello
[2020-05-08T11:32:02Z DEBUG cas::api::enclave::connection] V4(192.168.0.101:58584): Processing enclave hello message
[2020-05-08T11:32:02Z DEBUG cas::api::enclave::connection] V4(192.168.0.101:58584): SconeQuote is valid, it has not been tampared with.
[2020-05-08T11:32:02Z WARN  cas::api::enclave::connection] V4(192.168.0.101:58584): CAS is running in debug mode, simulation mode or w/o SCONE runtime: It will accept quotes from SCONE quoting enclaves running in debug mode
[2020-05-08T11:32:02Z DEBUG cas::api::enclave::connection] V4(192.168.0.101:58584): Hash of channel certificate: ec4f96d3faba2fdbbed2c7e1140f7df066e03440943eae7e9ab439c4262c075c74a8199b2b230a7760580c7a4799af34649e94a854709b6b08c61d6d7abac976, SGX reportdata: ec4f96d3faba2fdbbed2c7e1140f7df066e03440943eae7e9ab439c4262c075c74a8199b2b230a7760580c7a4799af34649e94a854709b6b08c61d6d7abac976
[2020-05-08T11:32:02Z DEBUG cas::api::enclave::connection] V4(192.168.0.101:58584): Secure channel terminates in enclave: Hash of channel certifacte matches SGX reportdata
[2020-05-08T11:32:02Z DEBUG cas::api::enclave::connection] V4(192.168.0.101:58584): SconeQuote is signed by known and trusted quoting enclave: true
[2020-05-08T11:32:02Z DEBUG cas::api::enclave::connection] V4(192.168.0.101:58584): Trust established into enclave integrity. Sending attestation complete message
[2020-05-08T11:32:02Z DEBUG eai] Sending message Complete to enclave
[2020-05-08T11:32:02Z DEBUG eai::convert] Waiting for more data before decoding protobuf message
[2020-05-08T11:32:02Z DEBUG eai::convert] Received length of next message
[2020-05-08T11:32:02Z DEBUG eai::convert] Next message will be 16 bytes long
[2020-05-08T11:32:02Z DEBUG eai::convert] Protobuf message received completely, interpreting it...
[2020-05-08T11:32:02Z DEBUG eai] Got message EnclaveMessage::ConfigRequest from enclave
[2020-05-08T11:32:02Z DEBUG cas::api::enclave::connection] V4(192.168.0.101:58584): In current state: Attested got message: EnclaveMessage::ConfigRequest
[2020-05-08T11:32:02Z DEBUG cas::api::enclave::connection] V4(192.168.0.101:58584): Received configuration request for config id javax2/hello
[2020-05-08T11:32:02Z DEBUG cas_backend::backend] Got request for service: hello in session: javax2. Attificate is: SCONE(
        SCONEAttificate {
            identifier: "791f84651890514eefe32ecfe54a75d95475636f61180a6ca1765cbbee0253dd",
            certificate: "-----BEGIN CERTIFICATE-----\nMIIBZjCB6qADAgECAhEA9vVqCQoOsEgtQ7Lo09QqNjAMBggqhkjOPQQDAwUAMA0x\nCzAJBgNVBAYTAkRFMCAXDTE4MDkxNzE5MzAwMFoYDzIwNTAwMTAxMDEwMDAwWjAA\nMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEoavE7JGhE5YqrBV979f8NTxUzGURrDE2\naU3vuZukWkioO8g+3Ec+YaVMu0N3CWy0hnwKD4nyCr7criLlXhvRaxQdJdp6d+4b\nMJXGtrgvcF4I1ZAb17FJOHoUD9Q5IVKSoxcwFTATBgNVHREEDDAKgghlbmNsYXZl\nADAMBggqhkjOPQQDAwUAA2kAMGYCMQCusCCDg8EKrrP75FL5kUShjoG28dWsAMVN\nDfv2InlX5ZCATrZ3YwSl8+MfV1Key0sCMQDMaA+jVF+bqr82C9iIYZhr4vXCWUhI\njYVnKAo+/GP+pwwvNE7XZwPTXbEPdRoyrbw=\n-----END CERTIFICATE-----\n",
            report: SgxReportBody {
                cpusvn: "02020000000000000000000000000000",
                miscselect: "00000000",
                attributes: SgxAttributes {
                    flags: 7,
                    xfrm: 27,
                },
                mrenclave: "7954791612f147832807e1290604c11a6cefb6ff12fb97a6b51530be5004aecb",
                mrsigner: "11c4e150b76c2b145f7fadb6c30455e1046b9e6fbb75b49c6e13341ad8acc5bd",
                isvprodid: 0,
                isvsvn: 0,
                reportdata: "ec4f96d3faba2fdbbed2c7e1140f7df066e03440943eae7e9ab439c4262c075c74a8199b2b230a7760580c7a4799af34649e94a854709b6b08c61d6d7abac976",
            },
            scone_qe_pub_key: a470cda5245af18ef5d8d3e1b30982a3bc74848376ca4fd03237c0f342e97239,
        },
    )
[2020-05-08T11:32:02Z INFO  cas::api::enclave::connection] V4(192.168.0.101:58584): Provisioned configuration javax2/hello
[2020-05-08T11:32:02Z DEBUG eai] Sending message Config to enclave
[2020-05-08T11:32:02Z DEBUG eai::convert] Waiting for more data before decoding protobuf message
[2020-05-08T11:34:50Z DEBUG eai::convert] Waiting for more data before decoding protobuf message
[2020-05-08T11:34:50Z WARN  cas::api::enclave] Failure while handling enclave connection: DisplayChain(Error(Msg("Connection closed"), State { next_error: None, backtrace: InternalBacktrace { backtrace: None } }))

And we see the following logfiles in the containers of LAS:

ubuntu@kmaster:~$ docker logs las_las_1
jhi[8]: --> jhi start
jhi[8]: <-- jhi start
aesm_service[11]: [ADMIN]White List update requested
aesm_service[11]: The server sock is 0x55c64a935e30
aesm_service[11]: [ADMIN]White list update request successful for Version: 73
[10000:INFO@04.05.2020/11:01:42] APP: Creating LAS target information message.
aesm_service[11]: [ADMIN]EPID Provisioning initiated
aesm_service[11]: The Request ID is f6131707e3714d359f33dca65dc72308
aesm_service[11]: The Request ID is 23c495fac0b94c8984546fbddc0a75f3
aesm_service[11]: [ADMIN]EPID Provisioning successful
[10000:INFO@04.05.2020/11:01:44] STARTER: SCONE QE has MRENCLAVE D9A05D04E07CD75F4251D41A4D7F0FCA63BF392A19BEDE5C69A5BB60A94E376F
[10000:INFO@04.05.2020/11:01:44] STARTER: SCONE QE has public key A470CDA5245AF18EF5D8D3E1B30982A3BC74848376CA4FD03237C0F342E97239
[10000:INFO@04.05.2020/11:01:44] STARTER: LAS is listening on 0.0.0.0:18766
[10000:INFO@04.05.2020/11:40:46] CONNECTION_HANDLER: Received attestation request
[10000:INFO@04.05.2020/11:40:46] CONNECTION_HANDLER: Sent quote success: 1, type: 3, size: 480
[10000:INFO@04.05.2020/11:40:46] CONNECTION_HANDLER: Received attestation request
[10000:INFO@04.05.2020/11:40:46] CONNECTION_HANDLER: Sent quote success: 1, type: 2, size: 1116
[01000:WARNING@04.05.2020/11:40:47] RECV_FUNC: Connection closed while reading message length.
[01000:WARNING@04.05.2020/11:40:47] CONNECTION_HANDLER: Unable to decode received message. Terminating connection!
...multiple log entries
[10000:INFO@08.05.2020/11:38:27] CONNECTION_HANDLER: Received attestation request
[10000:INFO@08.05.2020/11:38:27] CONNECTION_HANDLER: Sent quote success: 1, type: 3, size: 480
[01000:WARNING@08.05.2020/11:38:27] RECV_FUNC: Connection closed while reading message length.
[01000:WARNING@08.05.2020/11:38:27] CONNECTION_HANDLER: Unable to decode received message. Terminating connection!

Common errors

Missing LD_LIBRARY_PATH

If the system is started without the default LD_LIBRARY_PATH and the SCONE-Policy is missing its environment variable the output will be:

[SCONE|ERROR] src/syscall/execve.c:108:syscall_SYS_execve(): execve is only supported after a vfork (vfork is not active!) 
No error information
Error: trying to exec /usr/lib/jvm/java-1.8-openjdk/jre/bin/java.
Check if file exists and permissions are set correctly.

The solution is settings the LD_LIBRARY_PATH according to the posting session example given above:

LD_LIBRARY_PATH: "/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/.

Author: Hendrik