How is working the process of direct access to memory in non-SecureOS and SecureOS in trustzone systems

Question

Im not sure if I understand the full flow of CPU direct access to memory in ARM processors, I interested to know which part of memory access the cache (L1 and L2) ,DMA and MMU(or secure MMU) are participate. I'm not sure if I understand the process sending data from non-secureOS to SecureOS start from allocating shared buffer via DMA and writing data to to share buffer (between secureOS and non-secureOS) and sending.

Additional questions:

1. Why DMA needed to communicate between secure or non secure ? Why not possible to use via kernel buffer (kmalloc(), kzalloc(), get_page() and etc.)?
2. Generally, there is possible to CPU access to memory without DMA ? Does DMA must to participate ?
3. There is possible no-coherency between CPU(cache L1 or L2) to DMA ? For example: non-secureOS write own data to DMA buffer and send to secureOS. secureOS receive the buffer, non-secureOS change the buffer again without flushing (I think the changes keep in the cache) and finally secureOS read stale fake data from the cache

Answer 1

Everything with TrustZone is accomplished with the 'NS' bit that augments the BUS. For a TrustZone CPU, L1/L2/TLB (via MMU) need to be aware of the 'NS' bit. Caches and TLB are augmented with a 'NS' bit and are not accessible from the normal world if the 'NS' is clear.

I'm not sure if I understand the process sending data from non-secureOS to SecureOS start from allocating shared buffer via DMA and writing data to to share buffer (between secureOS and non-secureOS) and sending.

The secure/non-secure OS have several means to communicate. A DMA buffer is one way, but it is probably complex and would not be a normal mode. The most basic mechanism is the SMC instruction. This is trapped by monitor mode and accomplishes the same thing as a 'syscall'.

• Interpret ARM SMC calls
• ARM SMC Calling Convention
• SMC on StackOverflow

Another way is to map RAM as world shareable. Typically, this is done with a TZASC, but other Trustzone memory controllers may exist on a system. This is probably best 'bootstrapped' via the smc mechanics.

The use of a DMA controller could extend the world shareable memory buffer to off load CPU work load. However, I think this case is a little pathological and would never be done. Even faster than copying the memory via DMA is just to update the TZASC to make a buffer shareable. There is no copying.

1. Normal world reads 'secure memory' -> faults.
2. Normal world reads 'world shared memory' -> access as per normal.

The secure OS can flip the TZASC permissions during run time, if the device is not boot locked.

Why DMA needed to communicate between secure or non secure ? Why not possible to use via kernel buffer (kmalloc(), kzalloc(), get_page() and etc.)?

It is as detailed above. It requires world shareable memory.

Generally, there is possible to CPU access to memory without DMA ? Does DMA must to participate?

No DMA does not need to be involved at all. In fact, I wonder what made you think this is the case?

There is possible no-coherency between CPU(cache L1 or L2) to DMA ? For example: non-secureOS write own data to DMA buffer and send to secureOS. secureOS receive the buffer, non-secureOS change the buffer again without flushing (I think the changes keep in the cache) and finally secureOS read stale fake data from the cache

DMA and caches always have coherency issues. TrustZone doesn't add anything new. If you are using DMA, you need to have the MMU set that as device memory and it will not be cached.

Also, the DMA devices themselves are considered BUS masters. They can be TrustZone aware or some front-end logic if placed on them. In the first case, the controller with flip the 'NS' bit based on documented use patterns. For example a crypto device may present banked registers to normal/secure worlds. Depending on who accessed the device, the DMA will be performed with NS set or clear. For the 2nd case, another device/gasket sets up fixed access for the DMA. It is always either normal or secure access. This is often boot locked.

The DMA (and all hardware beside the CPU) are outside the scope of the CPU. The SOC designer and OEM have to configure the system to match the security requirements of the application. So different devices should map to normal/secure (or dynamic if required). The safest case is to fix these mappings and lock them at boot time. Otherwise, your attack surface grows in attacks against TrustZone.