HIPAA-Compliant AI: What the BAA Chain, ZDR, and Inference Architecture Require

TL;DR

• Confirming a vendor is "HIPAA eligible" means they'll sign a BAA, not that the surface your engineers open by default is covered by it. Consoles, playgrounds, standard API tiers without ZDR enabled, and beta features are explicitly excluded from BAA scope at every major provider.
• A BAA with your primary vendor doesn't extend to the upstream model providers it routes inference through, and each of those is an independent business associate under 45 CFR 160.103(3)(iii). If the vendor can't name every subprocessor in writing at the time of procurement, the chain has open links you can't see.
• Contractual ZDR includes safety-monitoring carve-outs that allow the vendor to retain data when a request is flagged, which apply to the clinical content most likely to trigger that flag. Architectural ZDR removes the storage path entirely, so the carve-out has nothing to act on.
• An agent routing across three model endpoints generates three separate retention profiles and three subprocessor relationships, none of which are covered by a ZDR term negotiated for the primary vendor. The BAA review at procurement didn't reach that topology because it wasn't visible when the review occurred.
• HIPAA's governing standard for PHI transmission is contact, not intent: a test fixture containing patient identifiers sent to an AI coding tool on a standard team plan constitutes an impermissible disclosure regardless of what the developer intended. The production BAA doesn't extend to the IDE, and that IDE is open every day.
• A standard API log recording of request and response isn't the audit trail 45 CFR 164.312(b) requires: a compliant record captures credentials, PHI scope, model, timestamp, and return value in a tamper-evident format across every intermediate agent action. If that record doesn't exist in exportable form, the deployment can't be defended in an OCR investigation.

What HIPAA-Compliant AI Means for Software Teams

The most common failure is a developer pasting patient data into a standard API endpoint because the vendor's security page said "enterprise-grade" and nobody read the scope clause on the BAA.

HIPAA doesn't regulate protected health information (PHI), and any system that creates, receives, maintains, or transmits PHI on behalf of a covered entity inherits the full scope of the Privacy Rule, Security Rule, and Breach Notification Rule, regardless of the technology it uses. Under 45 CFR 160.103, an AI vendor processing PHI operates as a business associate, which makes a signed Business Associate Agreement a legal prerequisite before any patient record flows to that vendor's infrastructure. What most engineering teams miss is that the BAA is the contract surface of a harder question: where does PHI travel, through which systems, under what controls, and who can read it at each step?

That question has grown more consequential; i.e., healthcare breaches exposed roughly 138.5 million individuals in 2025 alone, according to the HIPAA Journal's updated breach statistics. A developer on dev.to documented finding 50+ companies unknowingly building HIPAA violations into their AI workflows, not from negligence, but because the distinction between a HIPAA-eligible API tier and a consumer product sharing the same underlying model isn't written anywhere engineers naturally look.

This article covers the four places where healthcare AI deployments break down: the BAA chain, the zero-data-retention architecture gap, the inference-layer technical safeguards required by the Security Rule, and the agentic PHI perimeter that compliance reviews rarely reach. Where ORGN's Confidential Development Environment is relevant, specifically how hardware-enforced zero retention inside Intel TDX enclaves compares to contractual ZDR, that distinction is worked through with specifics.

The BAA Chain Problem: Why One Agreement Is Never Enough

A signed BAA with your primary AI vendor doesn't close the compliance perimeter if that vendor routes inference through upstream model providers it didn't name in the agreement. One missing link in the subprocessor chain renders every PHI-bearing request impermissible under 45 CFR 164.502.

When the Subprocessor List Is Longer Than the Marketing Page Admits

Under 45 CFR 160.103(3)(iii), subcontractors that create, receive, maintain, or transmit PHI are business associates in their own right. Each one requires BAA-equivalent terms before PHI can legally flow through it. Most AI SaaS products route inference through one or more upstream model providers, and most vendors' security pages don't publish a complete list of subprocessors. Engineering teams that confirm "the vendor is HIPAA eligible" and ship the feature without reviewing the subprocessor chain are carrying exposure they can't see.

The diagram below traces PHI through a real deployment: a covered endpoint under a verified BAA, routing into an upstream model that isn't, with an excluded surface sitting one step away from the path the team reviewed.

Every dashed line in that flow is a path the primary vendor BAA doesn't reach, and every one of those paths carries the same legal exposure as the covered endpoint above it.

The practical result shows up in enforcement data. OCR collected over $9.9 million in HIPAA settlements across 22 enforcement actions in 2024, with BAA deficiencies cited across a material number of those cases. The fix requires a specific question at procurement: not "will you sign a BAA" but "can you name every subprocessor that could touch our PHI and confirm each one operates under equivalent contractual terms?" A vendor that can't provide a written response is telling you something about the chain's completeness.

How BAA Scope Clauses Exclude the Surfaces Teams Use

Every major AI provider's BAA covers a narrower set of surfaces than their product marketing suggests, and the excluded surfaces are the ones developers reach for first. Anthropic's BAA applies to the first-party API under a zero-data-retention configuration, but explicitly excludes Console, Workbench, Claude.ai Free, Pro, Max, and the general Team and Enterprise plans. OpenAI's BAA covers the API on zero-retention-eligible endpoints and sales-managed ChatGPT Enterprise, but excludes ChatGPT Free, Plus, Pro, Team, and self-serve Business. Beta features at every major provider are categorically out of BAA scope until explicitly documented otherwise.

The scope clause is what determines whether your compliance posture is real or nominal. An engineering team that confirms "OpenAI is HIPAA eligible" and then routes PHI through the standard API without enabling ZDR has a BAA that doesn't cover how their system operates. Reading the scope clause, specifically which endpoints, plan tiers, and features are listed as in scope, is the decision that matters.

What "HIPAA Eligible" Means vs. HIPAA Compliant

"HIPAA eligible" is a vendor designation: the product tier can meet HIPAA requirements, and the vendor will sign a BAA. Compliance is the covered entity's construction atop that foundation. ZDR must be configured on every PHI-touching endpoint, training exclusion confirmed in the contract rather than the FAQ page, audit logging running, access controls scoped to the minimum necessary, and a risk analysis conducted specifically for the AI deployment. AWS Bedrock publishes a list of HIPAA-eligible services. Azure OpenAI falls under Microsoft's BAA. Google Vertex AI maintains a covered services document. In every case, the vendor provides eligibility. The team builds compliance.

A signed BAA with an eligible vendor protects you only if you operate within the eligible configuration, with the required controls active and the BAA covering the surfaces your PHI touches. "We have a BAA" and "we were HIPAA compliant" aren't the same statement, and OCR investigations have made that distinction expensive.

The CDE's model panel surfaces ZDR status at the point of selection, rather than burying it on a documentation page; every model is tagged before a session begins.

"Not confidential" label on Claude Fable 5 is the kind of distinction that doesn't appear in a vendor's security overview but determines whether a given model can legally handle PHI in a session.

Zero Data Retention at Inference: Architecture vs. Policy Promise

Contractual zero data retention is a legal commitment that PHI won't persist past the inference call. Hardware-enforced zero data retention is an architecture where persistence isn't technically possible. The gap between the two is where risk lives that contracts can't fully address.

The Gap Between Contractual ZDR and Technical Enforcement

Nearly every major AI vendor that offers ZDR does so on a contractual basis: your PHI won't be logged or retained, subject to safety-monitoring carve-outs. Those carve-outs rarely appear in the marketing copy. At most providers, requests flagged for policy violations can be retained for extended periods even under ZDR configuration. ZDR controls what's stored after inference. It doesn't control what the inference infrastructure itself can access during execution. Contractual ZDR is a promise the vendor can honor or breach. Architectural ZDR, in which session data is structurally discarded and no persistent storage path exists at the execution layer, removes the risk category for contracts that do not fully address it.

The state transition diagram below shows where contractual and architectural ZDR diverge: both start from the same inference memory state, but only one of them removes the host's ability to read it.

The contractual path passes through vendor runtime and policy controls before reaching a retention rule, and the safety-flag branch on the right is where the ZDR guarantee stops applying.

As ORGN confirmed in its April 2026 launch announcement, the CDE combines "hardware-backed confidential computing, a privacy-first retention posture for customer content" and "cryptographic attestation" tied to verified enclave execution. Inference running inside Intel TDX trusted execution environments executes within hardware-encrypted memory that the host OS and ORGN's own infrastructure can't read. Session data isn't policy-restricted; it's structurally unavailable. For healthcare engineering teams that carry HIPAA obligations into their development workflow, where PHI-adjacent code, schemas, and test fixtures regularly appear in AI coding context, that architectural distinction is the one a security review can verify rather than take on faith.

Why Agentic Workflows Break Standard ZDR Assumptions

ZDR on an inference endpoint applies to that endpoint. It doesn't apply to the intermediate state an agent generates between calls: working memory, tool call logs, outputs written to files, or context passed between subagents. A clinical documentation agent that reads a patient record, generates a draft note, and passes context to a summarization model for final formatting makes at least three inference requests, each with its own retention profile, across a data flow that may be invisible to whoever negotiated the BAA.

Most agentic frameworks don't expose the full inference chain to compliance teams. The team that approved the primary vendor relationship may not know that the agent routes certain task types through a secondary model, meaning the BAA review at procurement covered the front-facing vendor while the secondary model's subprocessor status was never evaluated. For regulated environments, the correct approach maps every model endpoint the agent can reach before PHI enters the pipeline, with BAA coverage confirmed at each node. ORGN's CDE captures agent actions, tool calls, and file-level changes in a tamper-evident log specifically because agentic sessions generate compliance-relevant events at every step, not just at the final output.

PHI in Development Environments: The Shadow AI Risk ZDR Doesn't Cover

Engineers building healthcare software bring PHI into development environments constantly without framing it as a compliance event. Test fixtures seeded with real patient records, stack traces containing identifiers, schema files describing PHI-bearing tables, inline comments referencing production incidents: all of this becomes AI coding context the moment a developer opens a session in an AI IDE.

Standard and Team plan tiers at GitHub Copilot and Cursor typically don't include a BAA. The team that negotiated an enterprise agreement for production AI may not have extended that coverage to the coding tools engineers use daily. ORGN's CDE addresses this gap by design: the same Intel TDX hardware isolation and zero-retention architecture that covers production inference also covers the development environment, under a single compliance boundary rather than a patchwork of separately negotiated enterprise tiers.

Technical Safeguards at the Inference Layer: What the Security Rule Requires

HIPAA's Security Rule at 45 CFR 164.302 through 164.318 establishes mandatory protections for ePHI across all systems that create, receive, maintain, or transmit it. These requirements extend to the inference API, the logging infrastructure, and the access control layer that scopes what data the model can reach. They don't stop at the database perimeter.

Encryption, Access Controls, and Audit Logging as Architecture

The Security Rule requires encryption in transit and at rest, role-based access controls, and audit mechanisms for all ePHI-containing systems. For AI deployments, those requirements produce three specifics that vendors often treat as optional: a complete log of every PHI-touching inference call (which model, who initiated it, what timestamp, what was returned), encryption extending into the execution environment and not only to the API gateway, and minimum-necessary access enforcement that scopes the model's data reach to what the specific task requires.

The audit logging requirement at 45 CFR 164.312(b) is where AI deployments fail most predictably. A standard API log records that a request was made and that a response was returned. A HIPAA-compliant audit log captures who made the request, under what credentials, against what PHI, and what the model returned, in an exportable and tamper-evident format, retained for the six-year minimum the Security Rule requires for compliance documentation. Multi-model deployments compound this: when different endpoints produce different log formats with different retention windows, assembling a unified audit trail for an OCR investigation means stitching together records that weren't built to connect.

TEE-Backed Execution and the Isolation Architecture Healthcare Needs

Standard cloud isolation separates tenants at the hypervisor or application layer. The cloud provider's infrastructure can, in principle, access workload memory. The isolation is real, but it depends on trusting the provider's internal access controls and its own personnel. Trusted Execution Environments using Intel TDX deliver a qualitatively different protection: hardware-level memory encryption that makes the enclave's contents cryptographically inaccessible to the host OS and the cloud provider alike.

For healthcare AI, this architecture matters because the threat model extends beyond external attackers. Vendor-side access to PHI during inference, through logging systems, abuse-monitoring pipelines, or debugging infrastructure, is an exposure that a contractual promise addresses only as well as the vendor honors that promise. Hardware isolation removes that dependency. ORGN's CDE runs sensitive inference inside Intel TDX enclaves and produces per-session cryptographic attestation: a verifiable record that the session ran inside a verified enclave, which a compliance team can export and include in audit documentation.

Where the Architecture Breaks: Agentic AI and the HIPAA Perimeter

Multi-model agentic systems don't have a single inference layer. They have a topology, a routing map through models and tools, each of which may represent a distinct business associate relationship. That topology is rarely visible to the compliance team that approved the initial vendor relationship, and the gap between the two views is where HIPAA exposure accumulates.

Subagent Routing and the Multi-Model PHI Exposure Problem

When an agentic framework routes between models- a planner, an executor, a summarizer- each endpoint processing PHI is an independent subprocessor under 45 CFR 160.103(3)(iii). The BAA chain must extend to each one. In practice, most agentic systems don't surface their routing topology to non-technical stakeholders, so the team that negotiated the primary vendor's BAA may have no visibility into which downstream models the agent calls. A planning model under BAA coverage routing context to a summarization model that isn't covered produces a disclosed PHI event, one that doesn't look like a disclosure from inside the development workflow.

The map below shows what the HIPAA perimeter looks like at runtime for a single agentic session: the BAA boundary covers the agent runtime and internal tools, but every external MCP server and API call the agent can reach sits outside it with its own coverage check required.

The X marks on Document Search, Scheduling API, and Email Service aren't hypothetical gaps, they're the tool paths an agent with PHI in context will call at runtime if nothing in the architecture prevents it.

The correct approach requires a compliance step before PHI enters the pipeline: map every model endpoint the agent can reach, confirm BAA coverage at each, and document the full subprocessor chain. When evaluating or building agentic healthcare AI, the question to ask of any vendor isn't "are you HIPAA compliant?" It's "can you produce the complete subprocessor list for every inference endpoint my PHI could reach, with written confirmation of BAA terms at each."

MCP Servers and Remote Tool Calls as PHI Leakage Vectors

Model Context Protocol servers allow AI agents to call external tools and APIs at runtime. When an agent operating on PHI calls a remote MCP server, that PHI exits the primary vendor's BAA boundary and enters third-party infrastructure the primary BAA doesn't cover. Both Anthropic and OpenAI explicitly document that remote MCP server calls fall outside their BAA scope. The architectural fix is clean. Either the agent's tool access stays restricted to local MCP servers inside the compliance boundary, or every remote MCP server that could receive agent calls during PHI-bearing sessions requires its own BAA before use.

This leakage is easy to miss because MCP calls happen at runtime rather than at configuration time. A developer who sets up an agent with access to a document search tool and a scheduling API may not consider that the same agent, operating on a PHI-bearing clinical record, will call both tools with that record in context. Runtime PHI leakage through tool calls doesn't feel like a disclosure from inside the development environment. It looks like normal agent behavior.

When the Audit Trail Disappears: Reconstructing Agent Actions Under HIPAA

HIPAA's audit controls requirement at 45 CFR 164.312(b) calls for mechanisms that record and examine activity in ePHI-containing systems. For an agentic AI, the audit record needs to capture every intermediate action: which files were read, which tools were invoked, what the agent reasoned before each action, and the sequence in which they occurred. A black-box agent that returns a finished document without any intermediate visibility isn't approvable in a regulated environment because the audit trail required by the Security Rule doesn't exist.

ORGN's CDE addresses this through full-session audit logging that captures streaming agent reasoning, tool call sequences, and file-level diffs in a tamper-evident format. For healthcare engineering teams, the difference between a vendor you can include in an OCR audit response and one you can't comes down to a single question: does the audit trail exist in exportable, tamper-evident form, or doesn't it?

The Confidential Observe panel maintains a structured record of every sandbox session, retaining status and task context over time, not just while the session is active.

For a compliance team reconstructing what an agent did with PHI during a stopped session, that persistent session index is where the investigation starts.

HIPAA Compliant AI in Regulated Engineering Teams: The Procurement Decision

HIPAA compliance for AI resolves to two questions that sound simple and aren't: does the BAA cover every surface and every subprocessor that touches PHI, and does an audit trail exist in a form you can defend under examination? A signed BAA with a vendor whose eligible configuration excludes the endpoints your engineers use daily isn't HIPAA compliance. It's documentation that creates the appearance of compliance while leaving the exposure intact.

The architectural hierarchy matters when choosing a vendor. Contractual ZDR is better than no ZDR. Hardware-enforced ZDR within a TEE is better than contractual ZDR because it removes a class of risk that contracts can't fully address: what the vendor's infrastructure does with PHI during inference before any retention policy kicks in. For healthcare engineering teams that need AI inside development workflows, not just production systems, that same architectural question extends to the IDE. A BAA covering the production API while leaving the coding environment unaddressed isn't a closed perimeter; it's two perimeters with a gap between them.

The procurement decision for HIPAA-compliant AI comes down to whether compliance rests on policy documents that carve out the exact scenarios that matter, or on architecture that makes those scenarios technically impossible. Policy-based compliance is defensible until OCR asks what happened inside the inference call. Architecture-based compliance answers that question with a cryptographic record rather than a contractual assurance.

FAQs

1. Does a signed BAA make an AI tool HIPAA compliant?

No. A BAA establishes the vendor's legal obligations for PHI, but compliance requires the covered entity to configure the deployment correctly, enforce access controls, maintain audit logging, and operate only within the BAA's listed scope. Vendors frame this as shared responsibility: the BAA covers the vendor's side; the covered entity owns the rest.

2. Can ChatGPT be used in a HIPAA-compliant way?

Not on consumer tiers; ChatGPT Free, Plus, Pro, Team, and self-serve Business have no BAA available, and using them with PHI is a violation regardless of how carefully the user operates. ChatGPT Enterprise (sales-managed) and the OpenAI API on zero-retention-eligible endpoints can be configured for compliant use once a BAA is executed and the deployment stays within covered surfaces.

3. What is zero data retention in a HIPAA context, and is it the same as no data storage?

ZDR is a vendor configuration term, not a HIPAA term. It means the vendor doesn't store prompt or response data beyond the inference request. It's not the same as hardware-enforced ephemerality. Most vendor ZDR agreements include carve-outs for safety-monitoring flags, so data can be retained when a request triggers a policy review. Hardware-enforced ZDR inside a TEE is the architectural version: retention isn't possible, only contractually restricted.

4. Do AI coding tools need HIPAA coverage if they're only used for development, not clinical care?

Yes, whenever the codebase or development context contains PHI. HIPAA applies to any system that creates, receives, maintains, or transmits ePHI, and development environments routinely hold PHI in test fixtures, database schemas, stack traces, and log files. The tool an engineer uses to write healthcare software is within HIPAA scope the moment it processes that material.