TrustOre: Side-Channel Resistant Storage for SGX using Intel Hybrid CPU-FPGA

Oh et al, ACM CCS 2020

Paper Summary

Memory-based side-channel attacks can use information like data-access patterns to deduce the contents stored in RAM, even in the presence of an enclave solution like Intel SXG. Previous work has suggest Oblivious-RAM as a mechanism for making access patterns to memory indistinguishable to an observer, and has demonstrated success in improving the security of such memory systems.

However ORAM backed solutions suffer from high performance impact and scalability problems due to the underlying data-structures that need to be maintained in this new mapping. TrustOre instead proposes a hybrid cpu-fpga solution, that places in-memory storage inside an external trusted component (FPGA). The FPGA retains its own memory related units that are physically isolated from untrusted software, preventing such software from observing the hardware directly. Evaluation of this approach showed significant performance gains at higher block sizes in comparison to related work, and demonstrated that trustOre is a practical solution that can be adapted to existing architectures.

This work1 makes the following contributions:

  1. A hybrid cpu-fpga architecture that extends the enclave trust model to an FPGA
  2. A design for providing trusted storage in an FPGA
  3. An implementation (System library/driver and Verilog design)
  4. Evaluation comparing against existing solutions in related work

The Problem

It is possible to observe the access patterns of memory via a memory-based side channel to deduce the contents held in memory, even if encryption is used, such as in the case of Intel SGX. Nullifying any confidentiality gained from using such an enclave scheme for securely storing data. Oblivious RAM (ORAM) solves this problem by attaching an interface to a CPU that acts as an intermediary to all memory i/o operations in a way that makes access patterns indistinguishable to the hardware. ORAM has been successfully applied to general data-structures like arrays and to subsystems like the file-system. While improving the security properties and resistance to memory-based side channel attacks, ORAM suffers from poor performance and scalability due to the management of extra data-structures required with this solution.

The Solution (or Approach)

TrustOre present a hybrid CPU-FPGA approach that transfers memory-management functions to an external FPGA physically isolated from untrusted sources. The external unit is considered secure from memory-based side-channel attacks as it does not hold generally exploited memory artifacts like caches and branch predictions, and instead maintains its own memory-related-units that only it has access to. Their solution has two components, a Trustlib driver in host memory, and a TrustMod that is loaded as a bitstream onto the FPGA fabric.

TrustMod is a secure storage service ran on the FPGA fabric. Attestation is supported by a public-key scheme that embeds the attestation private key inside manufacturer signed TrustMod bitstream loaded onto an FPGA. TrustLib can then verify the loaded bitstream by using the provided public key, as the corresponding private key is embedded in the signed bitstream and cannot be forged by the FPGA itself. TrustMod also contains On-Chip Memory for storing data-blocks and is supported by an On-Chip Memory Allocation Table (OCMAT) containing (block-id, enclave-id, size, base) tuple. This ensures only the originating enclave can perform operations (e.g. dealloc) on any memory stored on the FPGA (as the EID is derived from a dedicated MMIO address unique to each enclave).

TrustLib for the most part is considered as part of the host, and not the TCB, and exists to 1) verify the integrity of the TrustMod component at boot/run-time via attestation, 2) act as a mediator between user-code and the TrustMod by exporting a simple POSIX style interface. TrustLib will establish secure communication with TrustMod using a diffie-hellman key-exchange mechanism to derive a shared session key.

TrustOre prevents data-access pattern side-channels by all data-transfer operations are performed on a dedicated and fixed MMIO addresses. This means that regardless of execution context, the enclave will always access the secure storage in the same way, preventing a side-channel that monitors memory access patterns from discerning any information about the data-transferred.


Authors provided a (comparative) performance evaluation using an experimental method. They evaluated their workload against three previous studies:

  1. ZeroTrace -> data-array access
  2. Obliviate -> filesystem reads/writes
  3. Obfuscuro -> Generic data manipulation e.g. matrix multiplication

Across all experiments they found slower performance compared to native implementations. However comparison with related work showed comparable (but slower) performance at lower block sizes (approx < 2mb), but better performance higher block sizes. Performance scaling showed constant time performance in comparison to related work which often showed exponential growth in their experiments.

This difference at lower block sizes presumably reflects the overhead for PCI-e management at the beginning of an i/o operation in trustOre. The improved performance at higher-block sizes demonstrates the value provided by the trustOre service as memory can still be accessed and sent sequentially, unlike ORAM approaches which will still have to traverse an ever-growing tree data-structure.


  • no standard axis labels e.g. 1.E+06
  • good comparison with related work
  • modifications to existing work were made by authors, so some measured differences could be attributed to that

What Do You Think?

  • Overall pretty good paper
    • Like the hybrid design
    • Focus on real-world deployment
  • Susceptible to supply chain attacks as private-key is sent in bitstream to manufacturer and has to be signed by manufacturer
    • e.g. solarflare style attack
    • I understand this is out-of-scope but certainly impacts the practical/deployable focus that the authors put forward
  • One-way key-exchange validation for establishing secure channel
    • The TrustMod does not seem to verify that the TrustLib is valid
    • Session token re-use possible?
  • Doesn’t consider physical side-channels like thermal, power, or issues with RNG
    • Understand this is out-of-scope, but is the major source of side-channels on FPGAs


  • How does the FPGA solution protect against Rowhammer style attacks that leverage physical attributes of the platform
    • You could ask similar questions about heat and power (although they do comment that it is out-of-scope)
  • Could someone write their own TrustLib to masquerade?
  • Is there a mechanism for revocation of a public key?

  1. Paper Link - ↩︎

Nathan S Rutherford
Nathan S Rutherford
PhD Researcher in Information Security

I am based within the S3Lab at Royal Holloway, University of London, where my research centres around topics in systems security. My current work involves leveraging hardware capabilities for building dependable many-core operating systems.