Teaclave Service Internals

Teaclave Service is one of the most important abstractions in the platform. Basically, the Teaclave FaaS platform is the combination of different functional services and connected through trusted channels. The Teaclave services include authentication service, frontend service, management service, storage service, access control service, scheduler service, and execution service. They play different roles in the system.

To understand the design and internal implementation of these services, we need to discuss in these sections: RPC and protocol, app-enclave structure, and the attestation mechanism.

RPC and Protocols

We use Protocol Buffers (version 3) to define messages and RPC interfaces of the Teaclave services. For example, the authentication service has this definition.

message UserLoginRequest {
  string id = 1;
  string password = 2;
}

message UserLoginResponse {
  string token = 1;
}

service TeaclaveAuthenticationApi {
  rpc UserLogin (UserLoginRequest) returns (UserLoginResponse);
}

This means that the authentication service (for the API endpoint) has an RPC interface called "UserLogin", which takes a "UserLoginRequest" message with id and password inside (in string type) and reply a "UserLoginResponse" message with token inside (in string type).

With this definition, the build system will help to generate utility functions, traits, and structures for clients to send RPC requests, and for service to implement functions of handling requests. This is done by tonic_build.

For more protocol definitions for other services, please see proto files in the proto directory.

Service Implementation Structure

A service in Teaclave consists of two parts: the app (untrusted) part and the enclave (trusted) part. The app part is responsible for managing the service, launching and terminating the enclave part, while the enclave part is to serve RPC requests from clients (or other services) through trusted channels, execute logic, and process data in the trusted execution environment.

App (Untrusted)

Basically, the app part of a service does the followings:

• Load the runtime configuration from the config file.
• Create a service launcher: prepare the binder and set the serialized config as an input.
• Start the service enclave (ecall to the trusted enclave part).
• Misc: register signal handlers so that the app/enclave can respond to some signals.

Enclave (Trusted)

Typically, a service's implementation in the enclave part contains two important structs and one trait. Let's take the frontend service as an example.

• TeaclaveFrontendService (struct): Define properties or configurations along with the lifetime of the service. For example, the frontend service needs to hold clients (with established trusted channels) to communicate with the authentication service and management service.
• TeaclaveFrontendError (struct): Define errors that may occur in this service, authentication error, for example.
  • TeaclaveFrontend (trait Define): functions (requests) the service needs to handle. The trait will be automatically derived from definitions in the ProtoBuf file and can be imported from the teaclave_proto crate.

You will see some #[handle_ecall] annotations on functions and the register_ecall_handler macro to help with the function registration. The lifecycle of a Teaclave service consists of enclave initialized, service started, and enclave finalized, which will invoke the corresponding command handlers - InitEnclave, StartService, and FinalizeEnclave.

The start service function is the entry point of an enclave service. Here are steps to prepare and start serving requests.

• Initialize the attestation config and make a remote attestation.
• With the endorsed attestation report, initialize an attested TLS config.
• Initialize a TLS server with TLS config and listening address.
• If needed, initialize service endpoints this service wants to connect. For example, the frontend service needs to connect to the authentication service and management service.
• Start the service (with endpoint handlers) and begin to serve requests.

Here is a code snippet from authentication service in the enclave part:

let api_listen_address = config.api_endpoints.authentication.listen_address;
let attestation_config = AttestationConfig::from_teaclave_config(&config)?;
let attested_tls_config = RemoteAttestation::new(attestation_config)
    .generate_and_endorse()?
    .attested_tls_config()
    .ok_or_else(|| anyhow!("cannot get attested TLS config"))?;
let server_config = SgxTrustedTlsServerConfig::from_attested_tls_config(attested_tls_config)?.into();

let service = api_service::TeaclaveAuthenticationApiService::new(db_client, jwt_secret);

Server::builder()
    .tls_config(tls_config)
    .map_err(|_| anyhow!("TeaclaveAuthenticationApiServer tls config error"))?
    .add_service(TeaclaveAuthenticationApiServer::new(service))
    .serve(addr)
    .await?;

Topology

These services are communicating through RPC with remote attestation. Here is a topological graph illustrating connections between services.

clients => authentication <-+       +----> storage <----+
                            |       |                   |
clients => frontend ----------> management            scheduler <-- execution
                                    |
                                    +--> access_control


                                                  =>      api endpoint connections
                                                  -> internal endpoint connections

Attestation in Services

To explain the usages of remote attestation mechanism in services, we need to consider two different scenarios: 1) the service wants to serve RPC requests from clients, 2) the service wants to connect and send requests to other services.

For the first scenario, the endorsed attestation report is used for creating the trusted TLS server, so that clients can attest the service's report to verify the platform. In the meantime, if the service wants to attest clients (a.k.a., establishing mutual attestation), we need to get the accepted enclave attributes from the enclave info first to create the trusted TLS server. By this, the server can also attest clients' attestation reports and only accept expected connections.

You may find code like the following to get the accepted enclave attributes for mutual attestation:

let enclave_info = EnclaveInfo::verify_and_new(...)?;
let accepted_enclave_attrs: Vec<teaclave_types::EnclaveAttr> = AUTHENTICATION_INBOUND_SERVICES
    .iter()
    .map(|service| match enclave_info.get_enclave_attr(service) {...})
    .collect::<Result<_>>()?;

...

let server_config = SgxTrustedTlsServerConfig::from_attested_tls_config(attested_tls_config)?
    .attestation_report_verifier(
    accepted_enclave_attrs,
    AS_ROOT_CA_CERT,
    verifier::universal_quote_verifier,
)?;

For the second scenario, the report is used to create a trusted TLS channel so that the client can present its report when establishing the channel. Also, the server's report will be verified.

Customize a Standalone Service

For most cases, we suggest using the Teaclave platform as a whole for security and functionality concerns. However, you may want to customize a TEE service using Teaclave's existing capabilities under our service framework. For instance, the execution service can be used as a standalone TEE Python executor, and the storage service can be used as a secure database as well.

To customize Teaclave services as a standalone one, you need to first think about the interfaces exposed to clients, that is the definitions in protobuf. For example, for a key-value database, we have defined Get, Put and Delete interfaces.

Additionally, if you are using it as a standalone TEE service, the attestation mechanism needs to be "one-way attestation" accordingly. That is, only clients can establish trusted channels and attest the service's identity and platform status, but service cannot attest clients.