How to Choose a HIPAA-Compliant AI Infrastructure Provider

NoraLin 12 2026-10-16 04:27:29 Edit

A HIPAA-compliant AI infrastructure provider is a provider that signs a business associate agreement covering the specific AI services you use, implements the HIPAA Security Rule safeguards it is responsible for, and gives you the evidence needed to show your own compliance. There is no official HIPAA certification, so no provider is "HIPAA certified" and no provider can make a customer compliant on its own. The best provider for a healthcare AI project is the one whose agreement, controls, and evidence cover every place protected health information (PHI) will appear, including prompts, embeddings, model weights, and logs.

What HIPAA Requires of an Infrastructure Provider

Three parts of HIPAA define what to expect from a provider.

Requirement Source What it means for AI infrastructure
Business associate agreement (BAA) HIPAA Privacy and Security Rules (45 CFR 164.502(e) and 164.504(e)) Any provider that creates, receives, maintains, or transmits PHI for you must sign a BAA
Cloud providers are business associates even without keys HHS guidance on HIPAA and cloud computing (2016) A provider storing only encrypted PHI it cannot decrypt is still a business associate and still needs a BAA
Security Rule safeguards 45 CFR 164.308 (administrative), 164.310 (physical), 164.312 (technical) Access control, audit controls, integrity, authentication, and transmission security must be in place, split between you and the provider
Breach notification 45 CFR 164.410 A business associate must notify the covered entity of a breach without unreasonable delay and no later than 60 days after discovery

HIPAA does not require PHI to stay in the United States. HHS guidance allows offshore storage if the risks are weighed in the risk analysis. Many healthcare organizations still require U.S. residency as a risk decision.

The Security Rule Is Changing

HHS published a proposed update to the Security Rule on January 6, 2025. As of August 2026 it was still a proposal, with a final rule projected for July 2027 in the federal regulatory agenda. The proposal would, among other changes:

  • Remove the distinction between "required" and "addressable" specifications, making encryption of ePHI at rest and in transit mandatory with limited exceptions
  • Require multi-factor authentication for access to systems containing ePHI
  • Require an inventory of technology assets and ePHI data flows
  • Require at least annual penetration testing and regular vulnerability scanning
  • Require business associates to notify covered entities of security incidents within 24 hours
  • Require covered entities to verify, not only document, business associates' safeguards

None of this is enforceable yet. Still, choosing a provider that already meets these proposed requirements reduces rework if the final rule resembles the proposal.

Where PHI Appears in an AI System

AI systems put PHI in more places than a traditional clinical application. Each one must be covered by the BAA and the provider's controls.

Location Example What to check
Training and fine-tuning data Clinical notes used to fine-tune a documentation model Storage location, access control, encryption, and deletion
Prompts A clinician pastes a patient history into an assistant Whether the model service is in BAA scope, and where requests are processed
Retrieved context RAG retrieves prior encounter notes Permission-aware retrieval and logging of what was retrieved
Embeddings and vector indexes Notes converted to embeddings Treated as PHI unless properly de-identified
Model weights A model fine-tuned on PHI Access control and location; models can reproduce training data
Logs and traces Prompts and outputs stored for debugging Redaction, retention limits, and storage inside the BAA scope
Evaluation sets Real cases used to test accuracy Same controls as training data

De-identified data is not PHI. HIPAA allows two methods: Safe Harbor, which removes 18 specified identifiers, and Expert Determination, in which a qualified expert documents that re-identification risk is very small (45 CFR 164.514(b)). De-identifying training data reduces the scope of what must be protected, but prompts and logs from live clinical use usually remain PHI.

Provider Types and What Each Means for HIPAA

Provider type BAA pattern Your responsibility Watch for
Hyperscalers BAA covering a published list of eligible services Configure each eligible service correctly; avoid services outside the list AI services or deployment types that are not on the eligible list
Specialized GPU clouds Varies by provider and product Confirm whether a BAA is offered and which products it covers Self-serve instances and managed services may be treated differently
Managed private AI infrastructure BAA for a dedicated environment, with operations in scope Applications, users, and data governance Clear split of responsibilities in the shared responsibility matrix
Self-hosted on-premise No infrastructure BAA needed for your own staff Everything, including physical safeguards and 24/7 operations Staffing and evidence for audits

Evaluation Scorecard

Area Question Evidence to request
BAA scope Which exact services, GPU products, and support channels does the BAA cover? The BAA and a list of in-scope services
Shared responsibility Which Security Rule safeguards does the provider implement, and which are yours? A responsibility matrix mapped to 164.308, 164.310, and 164.312
Independent assurance Which audits cover the environment you will use? SOC 2 Type II report, HITRUST certification if offered, penetration test summary
Access control Who at the provider can access the environment, and how is it approved and logged? Privileged access procedure and sample logs
Encryption and keys Is ePHI encrypted at rest and in transit, and who holds the keys? Key management design
Audit controls Can you get logs of access to datasets, models, and endpoints? Log types, retention, and export method
Tenancy Is the hardware shared with other customers? Architecture description
Location Where are data, backups, and logs stored? Named facilities and DR locations
Incident notification How fast will you be notified of security incidents and breaches? Contract terms, compared with the 60-day limit and the proposed 24-hour incident standard
Subcontractors Which subcontractors handle PHI, and do they sign BAAs? Subprocessor list
Exit How is PHI returned or destroyed at contract end? Data return and destruction procedure

A provider that can answer every row with documents is a stronger choice than one that answers with marketing statements.

Worked Example: Responsibility Split for a Clinical Documentation Assistant

Safeguard Provider (managed private infrastructure) Healthcare organization
Physical safeguards for servers and facility Yes Review evidence
Encryption at rest and in transit Implements Approves design, holds keys if required
Infrastructure access control and logging Implements for administrators Implements for users and applications
Patch and vulnerability management of infrastructure Yes Applications and models
Permission-aware retrieval Provides platform capability Configures permissions from clinical systems
Risk analysis Supports with documentation Owns and documents
Workforce training and policies For its own staff For its own workforce
Breach notification Notifies the covered entity Notifies individuals and HHS as required

Where OneSource Cloud Fits

OneSource Cloud provides dedicated AI infrastructure designed for regulated healthcare workloads, with a HIPAA-ready infrastructure posture. Its AI for Healthcare solution runs on single-tenant GPU environments in U.S. data centers, including Richardson, Texas, with Managed AI Infrastructure for 24/7 operations. The OnePlus Platform, OneSource Cloud's AI orchestration platform, keeps model serving, notebooks, and logs inside the same dedicated environment. Healthcare teams should review BAA terms and the responsibility matrix as part of evaluation. For a control-by-control view, see the HIPAA AI infrastructure controls map and the HIPAA compliance checklist for AI infrastructure deployments.

FAQ

Which AI infrastructure provider is HIPAA certified?

None. HHS does not certify providers for HIPAA. Look for a provider that signs a BAA covering the services you use, documents its Security Rule safeguards, and provides independent evidence such as a SOC 2 Type II report or HITRUST certification.

Do I need a BAA with my GPU cloud provider?

Yes, if PHI will be stored or processed on its infrastructure. HHS guidance says a cloud provider is a business associate even if it holds only encrypted PHI and has no key.

Can I send PHI to an LLM API?

Only if the provider signs a BAA that covers that specific service and deployment type, and your risk analysis supports it. Otherwise, de-identify the data or run the model on infrastructure covered by your BAA.

Does HIPAA require data to be stored in the United States?

No. HIPAA has no explicit location requirement, but offshore storage must be weighed in the risk analysis, and many healthcare organizations require U.S. residency as policy.

Are embeddings of clinical notes considered PHI?

Treat them as PHI unless the source data was de-identified under Safe Harbor or Expert Determination. Embeddings and their stored text chunks can reveal the original content.

Summary

Choosing a HIPAA-compliant AI infrastructure provider is about coverage and evidence. The BAA must cover the exact services you use, the Security Rule safeguards must be split clearly, and every place PHI appears in the AI system must be inside that scope: training data, prompts, retrieved context, embeddings, model weights, logs, and evaluation sets. Ask for documents, not assurances. Prefer providers already meeting the proposed Security Rule changes, such as mandatory encryption, MFA, and 24-hour incident notification.

To evaluate a dedicated environment for healthcare AI, explore OneSource Cloud's AI for Healthcare solution.

Tags:
Previous: AI Infrastructure for Healthcare: How to Build HIPAA-Ready Private AI Environments
Next: Sovereign AI Cloud: Data Control for U.S. Enterprises
Related Articles
新模块