Auditing an AI Infrastructure Provider: 7 Security Controls to Verify

Auditing an AI infrastructure provider's security posture means verifying the controls that protect your data and models — isolation, identity, encryption, logging, and staff access — with evidence rather than marketing claims, because the provider's security failures become your incidents and your liability. Teams that accept a provider's security assurances without proof discover the gaps during a breach or an audit, when it is too late to switch providers.

For teams moving regulated, proprietary, or sensitive workloads onto external AI infrastructure, the security audit is not a formality. AI workloads introduce surfaces traditional cloud audits never covered: model weights, training datasets, checkpoints, prompts, and inference logs each carry confidentiality and integrity risks. A provider that secures storage but leaks model weights through shared GPU memory has failed the audit even if its storage controls look fine on paper.

This guide covers the seven control areas to verify, the evidence to demand for each, and the red flags that should stop a deal. It is the audit checklist used in vendor due diligence because it produces a verdict, not a feeling.

Why AI Infrastructure Security Is Different

AI infrastructure adds security surfaces that traditional cloud audits miss. Model weights are valuable intellectual property that must be protected at rest, in GPU memory, and during transfer. Training datasets often contain regulated or proprietary information whose exposure is a breach. Checkpoints can encode representations of training data. Prompts and inference logs can contain sensitive user content. And GPU memory is shared infrastructure that, without proper isolation, can leak data between workloads.

This means the audit must cover the AI-specific data path, not just the standard cloud controls. A provider can pass a generic cloud security review and still fail an AI-specific one, because the AI surfaces were never examined. Build the audit around both the traditional controls and the AI-specific ones.

Control Area 1: Isolation and Multi-Tenancy

Isolation determines whether your workloads are protected from other tenants on the same infrastructure. The strongest isolation is single-tenant dedicated hardware, where no other customer's workloads share your GPUs or hosts. Shared or multi-tenant environments rely on software and hardware isolation that must be verified, because isolation failures expose your data and models to other tenants.

Demand evidence of how isolation is implemented and tested. For GPU sharing, ask how memory is cleared between workloads, because residual data in GPU memory is a real leakage vector. For network isolation, ask how east-west traffic between tenants is controlled. For storage, ask how data is segregated and whether deletion is cryptographically or physically guaranteed. A provider that cannot explain its isolation model in concrete terms is one whose isolation has not been stress-tested.

Control Area 2: Identity and Access Management

Identity controls determine who can reach your data and models. Verify that access is authenticated, authorized by role, and logged. Demand evidence of role-based access controls with least-privilege defaults, multi-factor authentication for administrative access, and a clear model of which identities — yours and the provider's — can reach which resources.

The critical question for AI infrastructure is provider staff access. Does the provider's support or operations staff have access to your data, models, or consoles, and under what controls? Unbounded provider access is a common and serious risk, because it puts your sensitive data within reach of people outside your organization. Demand a clear answer on staff access scope, the controls that bound it, and the logging that records it. Private AI infrastructure with strict access boundaries typically offers a clearer and smaller staff-access surface than large shared clouds.

Control Area 3: Encryption and Key Management

Encryption protects data at rest, in transit, and increasingly in use. Verify encryption for every surface: storage, network traffic, checkpoints, and backups. The more nuanced questions are about keys. Who manages the encryption keys, where do they reside, and who can use them?

Demand evidence of key residency consistent with your data residency requirements, customer-managed key options if your policy requires them, and a key management plane that cannot be operated from outside your boundary. For AI workloads, also ask about encryption of model artifacts and whether GPU memory protections (such as confidential computing where available) are supported. Encryption that protects storage but leaves model weights unprotected in GPU memory is incomplete for AI workloads.

Control Area 4: Logging, Monitoring, and Audit Trails

Logging is how you prove what happened, both for your own investigations and for regulators. Verify that access to every regulated surface is logged, that logs are tamper-resistant, and that log storage respects your residency and retention requirements. Demand evidence of what is logged (reads, writes, administrative actions), how long logs are retained, and who can access them.

For AI workloads, ask specifically about logging of data movement events: checkpoint saves, model exports, dataset access, and inference log retention. These AI-specific events are where confidentiality breaches often originate, and a logging setup that captures only generic cloud actions will miss them. The audit should confirm that the evidence you would need after an incident actually exists.

Control Area 5: The AI-Specific Data Path

This is the control area traditional audits skip and AI audits must cover. Map the full data path your workload will follow — from dataset upload through training, checkpointing, model storage, inference, and log retention — and verify controls at each hop. The questions: Is the dataset isolated during training? Are checkpoints encrypted and access-controlled? Are model weights protected at rest and during transfer? Are inference logs redacted or retention-bounded when they contain sensitive prompts or outputs?

Each hop is a potential breach point. A provider with strong storage encryption but unbounded inference log retention fails this control, because the logs become a trove of sensitive content with no lifecycle controls. The audit must trace the data path end to end, not check controls in isolation, because a chain is only as strong as its weakest hop.

Control Area 6: Compliance and Attestations

Compliance attestations (SOC 2, ISO 27001, HIPAA, FedRAMP where applicable) are useful starting points but not endpoints. An attestation proves a provider was assessed against a standard; it does not prove the specific controls your workload needs are in place. Demand the attestation reports, read the scope, and verify the scope covers the services and regions you will use.

The gap to watch is scope. An attestation that covers the provider's core platform but not the AI-specific services, or that covers one region but not yours, leaves your workload outside the assessed scope. Treat attestations as evidence to read carefully, not as a blanket assurance. For regulated workloads, also confirm the provider will sign a business associate agreement or equivalent where required.

Control Area 7: Incident Response and Notification

Security incidents happen; the question is how the provider responds and how you find out. Verify the incident response process, the notification commitments (how fast you learn of an incident affecting your data), and the forensic support the provider offers. Demand the contractual notification timeline and the escalation path.

The red flag here is vague language. A provider that commits to "prompt notification" without a timeline, or that reserves broad discretion to define what counts as an incident, leaves you exposed to late or incomplete disclosure. For regulated workloads, late notification can itself be a compliance failure, so the timeline must be short and explicit.

Security Audit Checklist at a Glance

Red Flags That Should Stop a Deal

Certain answers should end the evaluation regardless of other strengths. A provider that cannot explain its isolation model in concrete terms has not stress-tested it. A provider with unbounded staff access to customer data treats your sensitive content as reachable by its employees. A provider whose logging does not cover AI-specific data movement cannot give you the evidence you would need after an incident. A provider whose attestation scope excludes the services or regions you will use is not assessed for your workload. A provider that commits only to "prompt" incident notification without a timeline leaves you exposed to late disclosure. Any of these is a reason to walk away or require remediation before signing.

FAQ

What evidence should I request from an AI infrastructure provider?

Request concrete evidence for each control area: the isolation model and how GPU memory is cleared between workloads; RBAC configuration and staff access scope with logging; encryption scope and key residency with customer-managed key options; access and data-movement logs with retention policy; a data path map showing controls at every hop; attestation reports with scope verification; and a contractual incident notification timeline. Marketing language is not evidence; ask for documents and configurations.

How is auditing an AI provider different from auditing a cloud provider?

AI providers add surfaces traditional cloud audits miss: model weights, training datasets, checkpoints, prompts, and inference logs, plus the GPU memory sharing model. A provider can pass a generic cloud security review and still fail an AI-specific one, because the AI data path was never examined. The AI audit must trace controls end to end across the data path, not just check standard cloud controls in isolation.

Are SOC 2 or ISO 27001 enough for AI infrastructure?

They are a useful starting point but not an endpoint. An attestation proves the provider was assessed against a standard; it does not prove the specific controls your workload needs are in place. Read the scope carefully: an attestation that excludes the AI services or your region leaves your workload outside the assessed scope. Pair attestation review with the control-specific evidence in this audit.

How do I verify provider staff access to my data?

Ask directly whether provider support or operations staff can access your data, models, or consoles, and under what controls. Demand the staff access scope, the controls that bound it (such as just-in-time access, approval workflows), and the logging that records it. Unbounded provider access is a serious risk; a clear, bounded, logged access model is what you want to see, and private infrastructure providers typically offer a smaller staff-access surface than large shared clouds.

What are the biggest red flags in an AI provider security audit?

The biggest red flags are an isolation model the provider cannot explain concretely, unbounded staff access to customer data, logging that does not cover AI-specific data movement, attestation scope that excludes your services or region, and vague incident notification commitments without a timeline. Any of these should stop a deal or require remediation before signing, because each represents a control gap that becomes your incident.

Summary

Auditing an AI infrastructure provider's security posture means verifying seven control areas with evidence, not assurances: isolation and multi-tenancy, identity and access (including provider staff access), encryption and key management, logging and audit trails, the AI-specific data path, compliance scope, and incident response. AI workloads add surfaces traditional audits miss, so the audit must trace controls end to end across the data path. Red flags — unexplained isolation, unbounded staff access, missing AI logging, scope gaps, vague notification — should stop a deal or require remediation. Teams that audit this way select providers whose security failures do not become their own incidents.

For teams that want a smaller, clearer security surface, private AI infrastructure with strict access boundaries typically offers a more auditable posture than large shared cloud footprints.