This blog aims to introduce the concept of Trusted Execution Environment (TEE) and how end users can leverage open source software to safely deploy applications that require handling confidential information.
Trusted Execution Environment (TEE) Overview
What is a TEE? What are the benefits of having a TEE?
TEE provides an isolated environment to ensure code/data integrity and confidentiality. Typical embedded systems running Linux or Android are exposed to a large number of security vulnerabilities in both the kernel and user space packages. Vulnerabilities can allow an attacker to gain access to sensitive information and/or insert malware. TEE adds an additional layer of security where code/data running on the TEE can not be accessed/tampered from the normal world OS (eg: Linux/Android). The software running on TEE (secure world) typically involves a tiny security oriented operating system (eg: OP-TEE OS) along with trusted applications. The trusted applications are meant to handle confidential information such as credit card PINs, private keys, customer data, DRM protected media, etc. and provide services to the normal world OS to make use of the confidential information without compromising it.
How is TEE implemented?
TEE requires both software and hardware (built into the processor) support.
On the hardware side, ARM based processors achieve TEE using TrustZone technology. TrustZone enables a single physical processor core to execute code safely and efficiently from both the normal world (Rich OS like Linux/Android) and the secure world (Security OS like OP-TEE). This allows high performance security software to run alongside the normal world operating environment. TrustZone implements a ‘state’ based memory and IO protection. i.e. when the processor is running in the secure state/context (secure world), it has a different view of the system and has access to memory/peripherals that normally can not be accessed from a non-secure state/context (normal world). The two virtual processors context switch via monitor mode when changing the currently running virtual processor.
Simplified hardware view of ARM TrustZone security
On the software side, there is a normal world OS (eg: Linux, Android etc) and a secure world OS (eg: OP-TEE, Trusty, QSEE, SierraTEE, etc.) both running in privileged mode. Similarly, there are user applications in normal world and trusted user applications in secure world both running in user mode. The secure world trusted applications/OS are meant to provide security related services to the normal world user applications.
Simplified software view of ARM TrustZone security
Choosing your secure world OS
Global Platform, a nonprofit organization, has developed TEE API and Framework specifications to standardize TEE and avoid fragmentation. There are various specifications available for TEE Client, Core, etc. that specify interactions between a Trusted Application and Secure World OS, a Trusted Application with another Trusted Application, a Client Application with a Trusted Application, and so on. This allows end users to switch a secure OS without having to change their Trusted Applications. When choosing your secure world OS, there are both commercial/proprietary and open source options. The choice of OS depends on the required features and level of support needed, but, it would be prudent to choose one that is compliant with Global Platform TEE specifications.
Open-source Portable TEE (OP-TEE)
OPTEE is a open source implementation of TEE. OP-TEE comprises of secure world OS (optee_os), normal world client (optee_client), test suite (optee_test/xtest) and Linux driver. The OS and client have a BSD 2-clause license and are Global Platform compliant. There are 28+ platforms/processors supported and actively maintained by Linaro. Below is a software architecture diagram of the OP-TEE. The product development team is responsible for developing a client application (CA) running on Linux and a trusted application (TA) running on OP-TEE. The CA uses the TEE client API to talk to the TA and avail secure services from it. The CA and TA used shared memory to pass data between each other.
OP-TEE software Architecture (Image Credit: Linaro)
Getting started with OP-TEE
Assuming the SoC you are using is already supported by OP-TEE OS, below are the options for building the software:
- Minimal OP-TEE build system: To help jump start running OP-TEE, some board maintainers (eg: Raspberry Pi 3 and TI AM43xx/AM57xx) provide a minimal build system that generates all the required images (bootloaders, OP-TEE OS, Linux kernel, OP-TEE client and a minimal RFS).
- Yocto Project build system: Include the meta-optee layer in your build and add the relevant packages to local.conf
- Manual: Cross-compile and build the various parts
Note: Various features in OP-TEE OS can be enabled/disabled by providing CFG_=<y/n> in the build command line.
For a custom board with a different memory size, you might need to adjust the secure (CFG_TEE_RAM_START, CFG_TA_RAM_START) and shared (CFG_SHMEM_START) memory locations in platform configuration file. Some platforms also allow the code to run from internal SRAM (CFG_WITH_PAGER).
Example OP-TEE memory map
In case the SoC you are using is not supported, contact your SoC vendor or DIY add support reaching out to OP-TEE maintainers, or seek commercial support (Timesys, WindRiver, …). You also have an option of testing out OP-TEE using a QEMU.
Bootloader support and boot flow
Ideally, the OP-TEE binary must be loaded as early as possible in the boot process (vulnerabilities in bootloaders running prior to OP-TEE could potentially compromise sensitive data). In a typical Linux boot (without TEE), the ROM bootloader loads/executes a 1st stage bootloader (eg: SPL, MLO, SBL1, FSBL) which then executes a 2nd stage bootloader (eg: U-Boot, LittleKernel) which executes the Linux kernel, all from a secure world context. On an ARMv7 based processor, the typical boot flow with TEE is: SPL loads OP-TEE and U-Boot, jumps to OP-TEE and once OP-TEE is done with initialization, it switches to non-secure context and jumps to U-Boot. The OP-TEE code will continue to reside in memory to provide secure services to the Linux kernel. On an ARMv8 based processor, the TEE boot flow involves an additional step of SPL loading ARM Trusted firmware, along with OP-TEE and U-Boot. SPL jumps to arm trusted firmware which later hands control to OP-TEE which in-turn jumps to U-Boot in non-secure context. On an ARMv8 platform, ARM Trusted firmware provides the monitor code to manage the switch between secure and non-secure world, whereas it is built-in to OP-TEE for ARMv7 platforms.
Note: OP-TEE needs to know the jump address to the non secure world (eg: u-boot or kernel load address). This can be provided either at build time (CFG_NS_ENTRY_ADDR) or SPL needs to configure ARM register (r1) with the load address before jumping to OP-TEE.
Different boot flows when using a TEE
Linux support
A Linux kernel driver for OP-TEE is available in kernel 4.12 or higher. If you are running a older version of kernel, then you will need to backport patches. You will also need to enable the driver following below steps:
- Set CONFIG_OPTEE=y in your kernel config
- Add device tree nodes for OP-TEE as shown here
- Alternatively compiling OP-TEE OS with CFG_DT=y can modify the dtb at runtime to add the required nodes
Troubleshooting OP-TEE issues
Once you have bootloader, Linux and OP-TEE OS running, you may potentially run into issues on your custom board. Below is a guide to understand and work around some common issues:
- Imprecise aborts: These are typically seen when a secure peripheral and/or memory is being accessed by the normal world OS. It is recommended to review the permissions of peripheral/memory regions.
- Memory map conflicts: The load address and runtime address of OP-TEE OS and kernel/dtb/ramfs need to be reviewed carefully, otherwise it could lead to issues like the kernel trying to relocate itself to the OP-TEE runtime region, etc.
- Clock setup: Linux kernel disables clocks to any unused peripherals but the peripheral might be used by the secure world OS which Linux is not aware of. One way to work around this is to include “clk_ignore_unused” as part of Linux kernel bootargs.
- Resource conflict: If a peripheral is being accessed by both normal and secure world code, there is no locking mechanism which could lead to conflicts. In the absence of a locking mechanism, it is recommended that only one of the world owns access to the resource and that it provide services to the other world to interact with the resource.
Trusted Applications (TA)
There are many tutorials available for trusted application development, hence this blog will concentrate on a high-level overview. For getting started with writing a CA and TA, refer to the hello world application.
There are 2 types of trusted apps: Dynamic and Pseudo/Static Apps
- Dynamic Apps reside in the normal world file system (RFS) and gets loaded into the secure world user space at runtime when a Linux client application wants to use it. The TAs are signed and OP_TEE OS verifies the signature before executing.
- Psuedo/Static Apps are typically built as part of the OP-TEE kernel and run in kernel mode. These apps mostly deal with providing services that involve controlling hardware which is difficult to implement in Dynamic App that runs in user space.
OP-TEE OS provides secure data storage facility for TAs. The data is stored either on the Linux filesystem (/data/tee) in encrypted/authenticated (AES-GCM) manner or on an eMMC RPMB partition.
Real world example
TEE can be used to reduce the hardware cost by replacing a dedicated security chip (key storage or crypto authentication) with a software solution.
To access the hardware features, many of the security chips provide an OpenSSL engine interface to user space applications. A similar model can be used by developing a TEE client app as part of OpenSSL engine to interface with the TA, thus minimizing any changes to user space applications. The Trusted Application will need to implement handlers for key management and crypto operations. OP-TEE OS includes libtomcrypt which provides various symmetric/asymmetric/elliptic curve cryptography functions. So the TA is mostly responsible for input validation and calling the appropriate OP-TEE core APIs. Below is an example architecture to achieve the same.
Example architecture for TEE protected cryptographic operations
Security considerations
There are system/platform level security considerations that need to be reviewed before deploying a TEE. Some of the common ones are covered below:
- JTAG: On a production unit, disable JTAG or limit JTAG to non-secure access only to ensure secure world data cannot be accessed/modified over JTAG.
- Secure boot: In order to ensure only authenticated code runs on the device, secure boot and chain of trust must be established. Refer to this blog for more details.
- Permission review: Some OP-TEE platforms do not explicitly set the permissions for non-secure world and defaults to allowing access to all peripherals/memory. Further, the registers configuring these permissions must also be explicitly set as secure access only in order to prevent normal world code from changing the permissions. Once the permissions are set appropriately, use devmem2/memtool from Linux user space to access secure memory regions, and verify a bus error is reported.
- Review use of keys: Secure storage depends on unique hardware key being provided to OP-TEE OS. Refer to this documentation to ensure it is setup correctly for your platform.
Conclusion
Using a TEE is a relatively inexpensive method to add an additional security layer to your device. The availability of open source software to leverage TEE makes deploying trusted applications easy. Apart from the added security benefits, it also has the potential to lower the hardware cost on platforms using a dedicated security chip/co-processor.
Timesys has experience adding support for new processors in OP-TEE and writing trusted applications.
Contact us to help you leverage TEE on your product.
About Timesys
Timesys has extensive experience with embedded system development and lifecycle management. Timesys has been instrumental in working with global leader semiconductor manufacturers with smart, quick and quality solutions for highly complex systems with accelerated product innovation and multiple product variants.
Thanks Akshay.
For answering my questions and providing me pointers to reach out for further information.
Its really helpful and thanks again
Best regards
Jayanth
Hello Akshay,
Firstly , a very good article as it strengthened my understanding on OP-TEE and TrustZone.
I have a question regarding Trusted Applications. Usually on the ARMv8 systems the bootflow is BL1–> BL2–> Trusted Firmware-A –> OP-TEE –> U-Boot –> Linux . So at the end of the boot flow , we are in normal world login prompt and Trusted OS (op-tee) running in parallel in the secure world.
Now how does the Trusted application need to developed and loaded in the memory such that it is not compromised.
Any idea on this ? or any tutorial on this?
I have referred to Helloworld example application where it is explained that we have a host and trusted application for Hello World. Both are stored in the rootfile system of the normal world in user/bin and /lib/tz folders.
To execute the trusted application, the client host application needs to invoke it.
So how is the SMC mechanism handled here. I mean when u run the helloworld client application how does the entire process happens . Could you explain me in simple as you have shown in the OP-TEE architecture. Thanks in advance.
Also the role of the TEE-Supplicant in the normal world ? what is the main use of this component in the OP-TEE.
Could you brief on this plz.
Hi Jayanth,
As you mentioned the TA typically resides in the rootfs of the normal world. In order to ensure it is not compromised, the TA needs to be signed and optionally encrypted. Details on how to do the same can be found here: https://optee.readthedocs.io/en/latest/building/trusted_applications.html#signing-of-tas
Regarding the host client needing to execute a TA, the nice thing about using OP-TEE is it is global platform specification and abstracts out the low level details of loading the application (smc calls associated with it etc). From a end user perspective what needs to happen is for the host client to call “TEEC_OpenSession” (https://github.com/linaro-swg/optee_examples/blob/master/hello_world/host/main.c#L56) with the UUID of the TA, which will result in the TA being loaded from the normal world filesystem automatically for you. The tee-supplicant, kernel driver and optee-os co-ordinate in the background to achieve this for you. If you are interested in more details of how things work behind the scene: https://optee.readthedocs.io/en/latest/architecture/trusted_applications.html#loading-and-preparing-ta-for-execution
Hope this helps,
Akshay
Hello Akshay,
Thanks for your answer.
So speaking in terms of security , the TA ( Both Client and the TA application ) both are stored in the /user/bin and /user/ta_armz/folders. Whenever we execute the client part the trusted application ( .ta ) will be invoked and then the optee-os will take care of authenticating and executing it. I think I summarise my understanding based on your reply. Hope I have understood correct.
Well I am curious , whether the trusted application on the op-tee os will stay alive during context switch mechanisms or every time the application needs to be loaded from the Normal world.
To be precise , assume I started a trustedapp which creates a key pair for me in the secure world. So the normal world client app invokes it and execues it in the secure world. Now during its execution , due to some reason the context switch needs to happen from secure to normal world. what happens now ?
Will the trusted application stillstays alive in the secure world and on the next context switch it continues from it was halted or it will be closed and needs to be loaded again on next iteration.
Thanks again for your reply . Hope you could help me in clearing this doubt as well.
Best regards,
Jayanth
The TA stays alive during context switch mechanisms till TEEC_CloseSession (https://github.com/linaro-swg/optee_examples/blob/master/hello_world/host/main.c#L101) is called. The secure world code has its own stack, so during a context switch it pushes its state to the stack and on return from normal world it pops the stack and resumes execution from where it left off. This is handled my the op-tee os / kernel drivers, the end user writing TA does not need to handle it.
Hi Akshay,
Thanks a lot for your quick reply and explaining it in a simple and better way.
But I have read at some point that OP-TEE does not have its own scheduler and is being scheduled by the normal world linux OS.
So assume based on some normal world timer the Linux OS schedules and runs OP-TEE. Now during the execution of the secure OS ( which includes running Trusted Apps ) as well . After this if the normal world timer expires , context switch happens from secure to normal world and the secure world state is pushed on to the stack and now control goes to normal world.
Am I correct ? Or OP-TEE will be running in parallel all the time and only the Trusted apps are scheduled from normal world ?
Could you share your thoughts on this please ?
Thanks in advance
Yes, OP-TEE does not have a scheduler support (yet) and execution of the secure world is driven from the normal world OS. While OP-TEE is running, if the normal world needs to execute code (eg: interrupt handling including schedule timer interrupts etc), then OP-TEE yields. Once normal world is done executing its code, the normal world calls back into the secure world to resume execution.
I think a better discussion place for such questions would be at the OP-TEE issues page or the mailing list. Typically most such questions are already answered if you go through the previous issues.
https://github.com/OP-TEE/optee_os/issues