Customer-Managed Key Controls for Enterprise AI Infrastructure

Customer-managed keys are encryption keys that a customer creates, controls, and can revoke in a key management service, while the infrastructure provider uses those keys to encrypt and decrypt data on the customer's behalf. In AI environments the control is valuable but narrower than most procurement teams assume, because AI workloads move data through stages where at-rest encryption does not apply.

The useful question is not whether a provider supports customer-managed keys. It is which assets those keys actually cover, at which moments data exists in plaintext, and what evidence you can produce when an auditor asks who could read your model artifacts.

This article maps the encryption surface of an AI environment, separates what key custody controls from what it does not, and lists the verification steps that belong in a provider review.

What Customer-Managed Keys Control in an AI Environment

Most infrastructure encrypts data at rest with keys the provider manages. Customer-managed keys change the custody model: the key material lives in a key management service under your control, and the provider requests use of the key rather than holding it. If you disable or delete the key, the provider loses the ability to decrypt the protected data.

That produces three concrete capabilities. You gain an independent revocation path that does not depend on the provider honoring a deletion request. You gain a key usage log that records decryption events. And you gain the ability to perform cryptographic erasure by destroying the key rather than tracking down every copy of the data.

These are meaningful controls. They are also frequently oversold, because the boundary of at-rest encryption in an AI stack is narrower than in a conventional application stack.

The Encryption Surface of an AI Workload

An AI environment holds far more distinct artifacts than a typical application, and each has a different encryption story. Mapping them is the first step of any key management review.

The pattern is consistent: key custody protects data while it is stored, and stops protecting it at the moment compute needs it. That is not a defect in the control, but it determines which risks it addresses.

Where Encryption Stops: GPU Memory and Compute

Training and inference require plaintext. Weights are loaded into GPU memory and stay there while the endpoint serves traffic. Training batches are decrypted, moved through host memory, and staged on device. Key management does not change any of this.

Three consequences follow for anyone writing a control narrative.

First, an operator with sufficient privilege on the host can potentially observe data in memory, regardless of key custody. This is why tenancy model matters as much as encryption: single-tenant hardware removes the class of risk where another customer shares the physical node. Confidential computing features on newer GPU platforms aim to narrow the host-level exposure, but availability and attestation requirements should be verified rather than assumed.

Second, memory hygiene between workloads is a separate control from encryption. When a job ends, GPU memory and node-local scratch must be cleared before the hardware is reassigned. Ask how that is enforced and how it is evidenced.

Third, key revocation does not reach running workloads. Disabling a key prevents future decryption; it does not evict weights already resident in GPU memory or terminate an active session. Revocation procedures need a matching workload termination step to be meaningful.

Custody Models and What Each One Buys

Vendors use overlapping terms for arrangements with materially different control profiles. Three patterns cover most deployments.

• Provider-managed keys: The provider generates and holds keys. Data is encrypted at rest, but custody and revocation stay with the provider. This is adequate for many workloads and inadequate when independent revocation is a requirement.
• Customer-managed keys: You create keys in a key management service and grant the provider usage rights. You hold revocation and receive usage logging. The provider still performs decryption inside its environment, so this establishes custody, not isolation.
• Hold-your-own-key arrangements: Key material never leaves your hardware security module, and the provider requests each unwrap operation externally. Control is strongest here, and so is operational fragility: a key service outage becomes a workload outage, which matters for long training runs.

Choose based on what you must be able to prove. If the requirement is "we can render data unreadable without provider cooperation," customer-managed keys meet it. If the requirement is "the provider is never in a position to decrypt," neither customer-managed keys nor hold-your-own-key arrangements achieve that during compute, and the honest architecture answer is dedicated hardware with strict access controls.

Operational Requirements Teams Underestimate

Key management introduces failure modes that AI workloads amplify. A distributed training job checkpointing every few minutes generates continuous key operations, so key service availability and rate limits become training reliability factors. A key service outage during a multi-day run can cost the run.

Rotation needs a defined behavior for existing artifacts. Most implementations rotate the key-encryption key and leave data-encryption keys in place, meaning historical checkpoints remain readable under the previous wrapping. If your control narrative claims that rotation limits exposure windows, confirm that the implementation matches the claim.

Access grants deserve the same scrutiny as the keys. A key policy that permits a broad provider service principal to decrypt is functionally close to provider-managed keys. Scope grants to specific roles and workloads, and review the key usage log the same way you review access logs — automated alerting on unexpected decryption patterns is more useful than a quarterly export.

These requirements are easier to satisfy when the compute environment is stable and single-tenant. On private AI infrastructure, storage tiers, node inventory, and access paths are known, so key scope can be mapped to specific hardware rather than to an abstract service boundary.

What to Verify Before Signing

A provider review should produce documented answers to the following, not marketing confirmation that customer-managed keys are supported.

1. Asset coverage: Which of datasets, checkpoints, deployed weights, vector indexes, logs, backups, and node-local scratch are encrypted under your key, and which are not.
2. Decryption actors: Which provider identities can invoke the key, under what conditions, and whether those invocations appear in a log you can read.
3. Revocation behavior: What happens to running training jobs and serving endpoints when the key is disabled, and how long until the effect is complete.
4. Teardown evidence: How GPU memory and local scratch are cleared between workloads, and what artifact proves it happened.
5. Backup and replica scope: Whether every copy — including snapshots and disaster recovery replicas — is protected under the same key, and where those copies are located.

For regulated deployments, these answers belong in the control documentation alongside residency and access evidence. Teams handling protected health information should align them with the broader control set described for healthcare AI infrastructure, since encryption is one control among several that reviewers examine together.

FAQ

Do customer-managed keys protect model weights during inference?

No. Weights are encrypted at rest but must be decrypted into GPU memory to serve requests, and they remain in plaintext there while the endpoint runs. Protecting weights during compute depends on tenancy isolation, host access controls, and memory hygiene between workloads rather than on key custody.

What is the difference between BYOK and customer-managed keys?

The terms are often used interchangeably. In practice, "bring your own key" usually means importing key material into the provider's key service, while customer-managed keys means the key stays in a service you control and the provider requests its use. Confirm which arrangement a specific product actually implements.

Does encryption slow down AI training?

At-rest encryption adds modest overhead to storage operations, which is normally not the bottleneck for training throughput. The more relevant risk is key service latency or rate limiting during frequent checkpoint writes, which can stall a job. Test at your checkpoint frequency rather than assuming.

Can we prove data deletion by destroying the key?

Cryptographic erasure is a recognized approach, but it only covers data encrypted exclusively under that key. Any copy encrypted under a different key, any plaintext export, and any derived artifact such as a vector index built from the data remain readable. Map derived artifacts before relying on key destruction as the deletion mechanism.

Are customer-managed keys required for HIPAA workloads?

Encryption is an addressable implementation specification rather than a fixed requirement for customer-managed keys specifically. Many regulated teams adopt them because independent revocation and usage logging strengthen the control narrative, but the compliance obligation is met through a documented set of safeguards, not a single feature.

Summary

Customer-managed keys give AI teams independent revocation, decryption visibility, and a cryptographic erasure path. They do not protect data during compute, they do not remove the provider from the decryption path, and they do not cover assets nobody mapped — most commonly vector indexes, prompt logs, and node-local scratch. Build the control narrative around a full asset inventory, verified decryption actors, tested revocation behavior, and documented teardown evidence.

Teams that need key custody alongside verifiable hardware isolation should evaluate providers on both. OneSource Cloud operates dedicated, single-tenant GPU environments in U.S. data centers with managed AI infrastructure operations covering workload teardown and access control. Request a security architecture review to map your encryption surface against your obligations.