.svg)
What HIPAA Establishes and Why Infrastructure Does Not Change It
The Health Insurance Portability and Accountability Act of 1996, signed by President Bill Clinton on August 21, 1996, created federal standards for protecting individually identifiable health information. The law applies to covered entities - health plans, healthcare clearinghouses, and healthcare providers that transmit health data in connection with regulated transactions - and to business associates, persons or organizations that perform services for a covered entity involving the use or disclosure of protected health information (PHI). The CDC summarizes the law's scope here.
The administrative simplification provisions in Title II produced five rules: the Privacy Rule, the Transactions and Code Sets Rule, the Security Rule, the Unique Identifiers Rule, and the Enforcement Rule.
None of those rules include a carve-out or relaxed standard based on where a system runs. A covered entity processing PHI on a private GPU cluster carries the same statutory obligations as one using a public cloud service. Infrastructure decisions do not determine whether HIPAA applies - they determine who holds each specific control obligation and how that responsibility is documented, tested, and enforced.
The Three Rules That Matter Most for AI Deployments
The Privacy Rule
The HIPAA Privacy Rule, effective April 14, 2003, governs what a covered entity may do with PHI - when it may use it, when it must disclose it, and when it must obtain patient authorization before disclosure. It gives patients the right to request copies of their health information, request corrections, ask for an accounting of disclosures, and receive notice of a breach.
For AI deployments, the Privacy Rule matters because training data, inference inputs, and model outputs can all contain PHI. If a covered entity passes patient records to an AI model - whether that model runs on a public cloud API or a privately hosted instance - the use of that data must conform to the permitted-use categories the Privacy Rule defines. Using PHI to train a commercial AI model without patient authorization is not permitted simply because the infrastructure is private.
The Privacy Rule also requires covered entities to apply the minimum necessary standard: only the PHI needed to accomplish the intended purpose should be used or disclosed, regardless of the computational environment.
The Security Rule
The Security Rule extends the Privacy Rule's protections specifically to electronically stored and transmitted PHI, known as ePHI. It requires covered entities to implement administrative, technical, and physical safeguards sufficient to ensure the confidentiality, integrity, and availability of ePHI.
Practical Security Rule requirements include access controls, data backups, contingency planning, and regular testing of those plans. The HIPAA Journal describes these as mostly "behind-the-scenes" measures that do not typically affect the patient experience but define the operational architecture of any HIPAA-covered system.
The Security Rule does not mandate a specific technology architecture. It does not prohibit public cloud and does not require private deployment. What it requires is that whatever architecture an organization selects actually implements the required safeguards and that the organization can demonstrate that implementation.
The Breach Notification Rule
If ePHI is accessed, acquired, used, or disclosed in a way not permitted under the Privacy Rule, the covered entity must notify affected individuals - and, in some cases, the Department of Health and Human Services and media outlets. Enforcement of the Privacy Rule, Security Rule, and Breach Notification Rule falls to the HHS Office for Civil Rights. The Centers for Medicare and Medicaid Services handles the Transactions and Code Sets Rules. HHS provides the full compliance framework here.
For AI systems, the Breach Notification Rule raises a specific question: if a model produces an output that exposes one patient's PHI to an unauthorized user, does that constitute a breach? That question is not answered by the infrastructure model alone - it depends on whether the safeguards the Security Rule requires were in place and whether they functioned correctly.
Business Associates and the Shared-Responsibility Question
When a covered entity uses an external vendor to process PHI - whether a public cloud provider, a private AI infrastructure company, or a managed services firm - that vendor almost certainly qualifies as a business associate. HIPAA defines business associates as persons or organizations performing services for a covered entity that involve the use or disclosure of PHI, including data analysis, data processing, and systems management.
Business associates must comply with the Security Rule and, depending on the nature of their service, certain Privacy Rule requirements. A covered entity is also responsible for ensuring that its business associates comply - typically through a Business Associate Agreement (BAA), a contract specifying each party's obligations.
In a public cloud model, the cloud provider typically offers a standard BAA covering the infrastructure layer. The covered entity retains responsibility for how it configures services, what data it places in which service, and whether application-layer controls meet Security Rule standards. The provider controls physical security, underlying hardware, and network infrastructure - but the covered entity configures data access policies, encryption key management, and logging within that environment.
In a private or dedicated AI infrastructure model, the division of responsibility can look quite different. If hardware is physically on the covered entity's premises, the organization may own all control layers: physical access, network segmentation, compute security, and application configuration. If the private deployment uses a dedicated environment hosted by a third party, the covered entity must understand exactly which controls the vendor owns and which remain the organization's responsibility.
The available HIPAA sources do not establish whether private dedicated infrastructure is easier or harder to keep compliant than public cloud. That depends on the organization's internal security capabilities, the vendor's contractual commitments, and the specific AI workload. Neither infrastructure model is inherently HIPAA-compliant or inherently disqualifying - both require deliberate implementation.
Four Compliance Dimensions Where Infrastructure Choices Have Real Consequences
The following areas are where the private-versus-public question becomes concrete. The HIPAA sources establish what each obligation requires; the infrastructure-specific comparison represents analysis of how those obligations apply across different control environments, not a regulatory determination.
1. Data Residency
HIPAA does not specify that PHI must be stored in a particular geographic location. However, some state laws impose stricter standards than HIPAA, and an organization can face obligations under a state's data protection rules based on where PHI originates - not just where the organization is headquartered. Texas, for example, applies its data protection laws to any organization that creates, maintains, processes, transmits, or receives health information about a Texas resident, regardless of whether the activity occurred in Texas.
In a public cloud deployment, data residency depends on which region the organization selects and whether the provider's BAA covers that region. In a private dedicated environment, the organization may have clearer physical control over where data sits - but only if the deployment is genuinely on-premises or in a contractually specified facility.
Organizations evaluating AI infrastructure should verify: In what physical locations can this deployment process or store PHI? Does the BAA cover those locations? Are there applicable state laws that create obligations the BAA or service agreement does not address?
2. Access Control Responsibility
The Security Rule requires that access to ePHI be limited to authorized users and systems - in practice, identity management, role-based access, and session logging across the full stack.
In a public cloud model, the provider typically handles physical and hypervisor-layer access, while the covered entity configures identity and access management policies for its own workloads. The risk is misconfiguration: the controls exist, but the organization must apply them correctly.
In a private or on-premises AI environment, the organization may control more layers of the stack, enabling finer-grained access policies - but also bearing more operational responsibility. If the private deployment is managed by a third-party vendor, the BAA must specify exactly which access control obligations the vendor owns and which the covered entity retains.
The practical question is not which model is "more secure" - it is which model the organization can operate correctly and audit reliably.
3. Audit Logging Ownership
The Security Rule requires that covered entities implement hardware, software, and procedural mechanisms to record and examine activity in information systems that contain or use ePHI. Audit logs are also foundational to the Breach Notification Rule - without them, an organization may not be able to determine whether an impermissible access occurred, who was affected, or when a breach began.
Public cloud platforms typically generate extensive audit logs at the infrastructure level. The covered entity must ensure those logs capture the specific activities HIPAA requires and that they are retained appropriately and accessible during a breach investigation.
In a private or on-premises environment, the organization is generally responsible for configuring and retaining its own audit infrastructure. Dedicated hosted environments may or may not include logging services, depending on the vendor agreement. Organizations should ask: Who generates logs for this environment? Who retains them, for how long, and in what format? Who can access them during an investigation?
4. Breach Response and Vendor Coordination
The Breach Notification Rule creates time-sensitive obligations. Covered entities must notify individuals without unreasonable delay and no later than 60 days after discovery. If a breach affects 500 or more individuals in a state or jurisdiction, the covered entity must also notify the HHS Secretary and prominent media outlets in that area.
When PHI lives in a vendor-managed environment - public cloud or private dedicated - the covered entity's ability to detect, investigate, and respond to a breach depends partly on the vendor's cooperation. A BAA should specify the vendor's obligation to notify the covered entity of a suspected breach, provide access to logs and affected records, and support the investigation.
The critical question is whether the BAA includes explicit breach response obligations and timelines sufficient for the covered entity to meet its own notification deadlines - regardless of whether the infrastructure is public or private.
State Laws and Multi-Jurisdiction Risk
HIPAA sets a federal floor, not a ceiling. States may impose stricter retention periods, stronger data protection standards, or broader definitions of covered activity. Because some state laws apply based on where data subjects reside rather than where the organization operates, an AI system processing health data from patients across multiple states may face overlapping legal obligations.
Private infrastructure does not simplify this multi-jurisdiction exposure. An on-premises server in one state may still process PHI subject to the laws of another state if the data relates to residents of that state. As the HIPAA Journal notes, some state laws do not distinguish between covered entities and business associates - any organization engaging in a covered activity is treated as a covered entity under those frameworks. Organizations should map their patient populations and data flows before assuming that a particular infrastructure choice reduces regulatory complexity.
What to Establish Before Deployment
Because the HIPAA regulatory sources do not compare compliance pathways across infrastructure models, the following are framed as due-diligence questions rather than resolved conclusions. Each requires vendor documentation and, for high-risk deployments, qualified healthcare privacy counsel.
- Will the vendor sign a HIPAA-compliant BAA covering the specific AI services and data flows in scope?
- Which Security Rule safeguards does the vendor implement and document, and which remain the covered entity's responsibility?
- Where, physically, will PHI be processed and stored - and does that location create state-law obligations beyond HIPAA?
- What access control mechanisms does the vendor provide, and how does the covered entity configure and audit them?
- Who owns audit log generation, retention, and retrieval, and in what format are logs produced?
- What is the vendor's contractual obligation when a suspected breach occurs - notification timeline, evidence preservation, investigation support?
- Does the vendor's environment support the Security Rule's contingency planning requirements, including data backup and recovery testing?
None of these questions have generic answers. A public cloud provider's answers and a private infrastructure vendor's answers may be structurally different, but neither is automatically superior - what matters is whether the answers satisfy the Security Rule's requirements for the specific workload.
Conclusion
HIPAA's Privacy Rule, Security Rule, and Breach Notification Rule apply to covered entities and their business associates regardless of whether AI systems run on public cloud or private dedicated infrastructure. The law does not mandate a specific architecture; it mandates specific safeguards. What changes across infrastructure models is the allocation of responsibility for those safeguards - who configures access controls, who generates and retains audit logs, who responds to a breach, and how all of that is documented in a Business Associate Agreement.
Public cloud deployments typically involve a shared-responsibility model where the provider controls physical and infrastructure layers and the covered entity configures application-layer safeguards. Private or dedicated infrastructure may shift more control layers to the organization or to a specialized vendor, requiring more precise contractual definition of each party's obligations. Neither model is inherently more compliant. Both require that the specific safeguards the Security Rule demands are implemented, documented, tested, and auditable.
Organizations evaluating AI infrastructure for PHI workloads should conduct BAA review, map data residency obligations including applicable state laws, and clarify audit log ownership and breach response timelines before deployment - and should involve healthcare privacy counsel in that review.
Frequently Asked Questions
Does HIPAA require organizations to use private infrastructure instead of public cloud for AI workloads?
No. HIPAA does not specify a required infrastructure model. The Security Rule requires covered entities to implement administrative, technical, and physical safeguards to protect ePHI, but it does not prohibit public cloud deployments and does not require private or on-premises infrastructure. Both models can satisfy the Security Rule if the required safeguards are implemented, documented, and testable.
If a third-party AI vendor processes PHI, must they comply with HIPAA?
Yes. A vendor performing data processing, analysis, or management services for a covered entity using PHI qualifies as a business associate under HIPAA and must comply with the Security Rule and certain Privacy Rule provisions. The covered entity must obtain a signed Business Associate Agreement from the vendor and remains responsible for ensuring the vendor's compliance with the agreement's terms.
How does HIPAA's minimum necessary standard apply to AI model training?
The Privacy Rule's minimum necessary standard requires that covered entities use or disclose only the PHI needed to accomplish the intended purpose. Applied to AI model training, this means an organization should evaluate whether full patient records are necessary for the training objective or whether de-identified or limited data sets would suffice. Using more PHI than necessary - even within a technically secure environment - may not satisfy the Privacy Rule's requirements. Organizations should consult healthcare privacy counsel before using PHI in AI training pipelines.
Sources
