Press play to start listening
Modern AI runs on shared, high-performance infrastructure that processes enormous volumes of sensitive data and valuable model weights. In these environments, capacity is often swapped rapidly between different customers, which raises a hard question. How do you keep data and models private while they are being computed on hardware you may not fully control?
Encryption already protects data when it is stored and when it moves across a network. The stubborn gap has always been data in use, the moment it is loaded into memory and processed. Confidential computing exists to close that gap.
This article surveys how confidential computing works across CPU and GPU systems, why heterogeneous setups are the current frontier, and where the real boundary of trust actually sits.
The Three States of Data and the Hardest One
Security teams usually think about data in three states. Data at rest lives in storage. Data in transit moves across networks. Both are routinely encrypted, and the tools to do so are mature.
Data in use is different. To compute on data, a system has to decrypt it into memory, where it becomes readable to anyone with sufficient privilege. In a multi-tenant AI data center, that exposure is a serious risk, because a compromised or malicious layer of the stack could read another tenant’s data or extract a model.
Protecting data in use is the problem confidential computing was built to solve.
What Confidential Computing Actually Is
Confidential computing is a hardware-based approach that runs sensitive computation inside a protected area called a Trusted Execution Environment, or TEE. A TEE isolates the code and data inside it from everything else on the machine, including privileged software such as the operating system and the hypervisor.
Inside a TEE, memory is encrypted and only authorized code can see the plaintext. The goal is to shrink the set of components you have to trust, so that even an attacker who controls the host software cannot simply read what the workload is processing.
Trusted Execution Environments on the CPU
The CPU side of confidential computing is the most established. Several technologies take different routes to the same aim.
Intel SGX creates small enclaves at the level of an individual process. Intel TDX raises that idea to the level of an entire virtual machine, creating isolated trust domains. AMD offers SEV and the more recent SEV-SNP, which encrypt virtual machine memory and add integrity protection. On the Arm side, TrustZone splits the processor into a secure world and a normal world, while the newer Confidential Compute Architecture introduces isolated realms.
The implementations vary, but the principle is consistent. Reduce the trusted computing base and keep data encrypted while it sits in memory.
The GPU Problem
There is one catch for artificial intelligence. Training and inference run on GPUs, not on CPUs. A CPU-only TEE leaves the GPU outside the protected boundary, which is precisely where the model and the data are actually processed.
Recent hardware addresses this directly. NVIDIA’s Hopper generation, including the H100, introduced confidential computing on the GPU itself. That allows data and model weights to remain protected while the GPU performs the heavy computation, rather than forcing teams to choose between performance and privacy.
This shift matters because it brings the accelerator, the most valuable and most exposed part of an AI system, inside the circle of trust.
Extending Trust Across a Heterogeneous CPU and GPU System
The genuinely difficult part is joining a CPU TEE and a GPU TEE into a single, coherent trust boundary. This is what makes heterogeneous confidential computing the active area of research.
Data has to travel between the CPU and the GPU across links such as PCIe or NVLink, and it must stay protected the whole way. That usually involves encrypting the transfers and using secure staging buffers so plaintext is never exposed on the bus. The CPU and GPU also need to authenticate each other, and the entire path has to hold up against an attacker who controls the host.
Coordinating two different processors, each with its own protection model, into one continuous guarantee is the core challenge the field is working through.
Attestation: Proving the Environment Can Be Trusted
None of this means anything unless the party sending in the data can verify what it is sending into. That verification is called remote attestation.
Before releasing sensitive data or model weights into a TEE, a relying party asks the hardware for proof that the environment is genuine and is running the expected, unmodified code and firmware. The hardware produces a signed measurement of that environment, which is then checked against known-good values. If the measurement does not match, trust is withheld.
Attestation is the linchpin of the whole model. Without it, an enclave is only a claim. And here is the detail that is easy to miss. Attestation measures firmware and low-level components, which means the entire scheme depends on the integrity of the hardware and firmware sitting underneath it.
Threat Models and Honest Limits
Confidential computing is designed to defend against strong adversaries, including a compromised operating system, a malicious hypervisor, or even someone with physical memory access. That is a high bar, and it is a real advance.
It is not absolute, though. Side channel attacks, which infer secrets from timing, power draw or microarchitectural behavior rather than reading memory directly, remain a persistent concern. There is also a performance cost to memory encryption and secure data movement, which teams have to weigh against the protection gained.
Above all, every TEE quietly assumes that its own firmware and hardware root of trust are sound. If that assumption fails, the guarantees above it fail with it.
The Foundation Everything Else Depends On
This is the point the survey keeps returning to, and it is the one AI data center operators cannot afford to skip. A TEE’s protections are only as strong as the firmware and hardware beneath them.
If a GPU’s firmware has been tampered with, if a counterfeit component slips through the supply chain, or if firmware is not properly sanitized between tenants on shared infrastructure, then the root of trust that attestation relies on is quietly undermined. Standards bodies have flagged exactly these issues.
NIST SP 800-223 highlights attacks on critical hardware components, the supply chain risk created by rapidly changing firmware, and the challenge of validating firmware between task runs on shared compute. OpenAI’s own guidance for secure AI infrastructure lists trusted computing for accelerators and tenant isolation among its core practices.
This layer is also the one most security tools overlook. A great deal of attention goes to AI software and model risks, as the OWASP Top 10 for large language models shows, while the hardware, components, and supply chain beneath them receive far less scrutiny.
According to Eclypsium, an AI data center security provider, while many vendors focus on software supply chain security, far fewer address supply chain security for the IT infrastructure itself. Rather than working at the level of endpoint detection or conventional vulnerability management, the platform secures infrastructure below the operating system, providing continuous visibility and monitoring of firmware, hardware, and low-level components across servers, network devices, and endpoints.
For AI data centers, that translates into concrete steps. Teams can inventory and validate the integrity of GPUs before they are installed or leased to a new customer, verify that firmware and configurations are known-good, and detect counterfeit or tampered components before anything is deployed.
Shared GPU resources can be re-verified between customers, so one tenant cannot leave a compromised firmware state behind for the next. When hardware reaches the end of its life, it can be scanned before disposal to confirm no sensitive data remains.
In other words, it works on the foundation that confidential computing takes for granted.
Where This Is Heading
The direction of travel is fairly clear. Expect broader standardization across vendors so that attestation and confidential computing are not locked to a single hardware line. Expect stronger support for multi-GPU and multi-node workloads, since real AI training rarely fits on one accelerator. Expect continued work on side channel defenses and on reducing the performance cost of protection.
The deeper trend is convergence. Protecting data in use and protecting the firmware and hardware that process it are turning out to be two halves of the same problem. A signed attestation is only meaningful if the silicon and firmware behind it can genuinely be trusted.
The Bottom Line
Confidential computing has come a long way, from process-level enclaves on the CPU to full protection on GPUs and now to the harder work of securing heterogeneous CPU and GPU systems end to end. For AI data centers handling sensitive data and costly model weights, it is becoming a baseline expectation rather than a luxury.
But the model only holds if its foundation holds. Trusted execution and attestation rest on the assumption that firmware and hardware are exactly what they claim to be. Securing that layer is what makes every protection above it believable, which is why it belongs at the center of any serious AI infrastructure security plan.
(Photo by Igor Omilaev on Unsplash)
