Do you need a BAA to use an LLM with patient data?

Only if a third party handles the PHI. HIPAA requires a business associate agreement under 45 CFR 164.502(e) when an outside entity creates, receives, maintains, or transmits PHI on a covered entity's behalf — which is what happens with every cloud AI API. If the model runs entirely on infrastructure the covered entity owns and PHI never leaves its network, there is no business associate and no BAA to sign.

Is ChatGPT HIPAA compliant?

Consumer ChatGPT is not — OpenAI does not sign BAAs for consumer products, and inputs may be retained and used for training. Some enterprise AI offerings will sign BAAs, which makes use permissible but leaves PHI on vendor infrastructure, subject to vendor retention, vendor breach exposure, and vendor legal process.

Can de-identified data be used with cloud AI without a BAA?

Yes, if it genuinely meets the HIPAA de-identification standard in 45 CFR 164.514 — Safe Harbor removal of 18 identifier categories or formal expert determination. In practice, clinical free text is notoriously hard to de-identify reliably at scale, and a single missed identifier in a prompt stream is a reportable problem. De-identification pipelines reduce risk; they do not eliminate it.

What is the safest LLM architecture for a hospital or clinic network?

A fully on-premise deployment of open-weight models — for example GLM-5.2 for language tasks and Qwen3-VL for document and image understanding — inside the network where the EHR already lives, with role-based access and audit logging. PHI never leaves the covered entity's control, so there is no business associate, no cross-border transfer, and no vendor breach surface.

HIPAA-Compliant LLMs Without a BAA: 5 Architectures Ranked

The fastest way to make an LLM "HIPAA compliant" is to remove the reason HIPAA's vendor rules apply at all. A business associate agreement is required — under 45 CFR 164.502(e) — when a third party creates, receives, maintains, or transmits protected health information on a covered entity's behalf. Every cloud AI API does exactly that, which is why the standard advice is "get a BAA." But a fully on-premise LLM deployment involves no third party: the model is a static file of open weights running on hardware the covered entity owns, inside the network where the EHR already lives. No outside entity ever touches PHI, so no business associate exists and no BAA is required. Below are the five architectures healthcare organizations actually use, ranked from weakest to strongest — with the honest trade-offs of each.

What are the five architectures for using LLMs with PHI?

#	Architecture	BAA required?	PHI leaves your control?	Residual risk
1	Consumer AI tools (ChatGPT and similar)	Not available	Yes — retained, possibly trained on	Non-compliant for PHI; shadow-AI reality
2	Cloud API + de-identification pipeline	No, if 164.514 is truly met	Only if de-identification fails	Free-text re-identification leakage
3	Cloud API with a signed BAA	Yes	Yes — vendor infrastructure	Vendor retention, breach, legal process
4	Dedicated/private cloud AI (single-tenant, BAA)	Yes	Yes — vendor-operated	Same custodian problem, better isolation
5	Fully on-premise open-weight deployment	No — no business associate exists	No — zero egress	Yours to manage under the Security Rule

1. Consumer tools: the shadow-AI floor

Consumer AI products do not sign BAAs and may retain and train on inputs; using them with PHI is a straightforward violation. The operational problem is that clinicians use them anyway when no sanctioned alternative exists — security-industry surveys consistently find roughly a quarter of employees admit pasting confidential work data into public AI tools. The fix is not another policy memo; it is an approved tool good enough that nobody routes around it.

2. De-identification pipelines: better than their reputation, worse than their promise

If data meets the de-identification standard in 45 CFR 164.514 — Safe Harbor removal of all 18 identifier categories, or expert determination — it is no longer PHI, and no BAA is needed for cloud processing. The catch is clinical free text: names, dates, MRNs, and rare-condition details hide in narrative notes, and a de-identification model that is 99% accurate still leaks identifiers at scale. Workable for research corpora; fragile for live clinical workflows like ambient scribing, where the raw conversation is inherently identified.

3–4. Cloud with a BAA: permissible, not painless

A signed BAA makes cloud AI use lawful, and single-tenant "private" cloud offerings improve isolation. But the structure is unchanged: PHI sits on vendor infrastructure, under vendor retention policies, inside vendor breach statistics, and subject to legal process served on the vendor. The covered entity's compliance posture becomes a supply chain of contracts to police — BAAs, subcontractor BAAs, audit rights — and HHS breach-notification duties still land on the covered entity when a vendor fails. Contractual scaffolding manages the risk; every layer of it is something to monitor forever.

5. On-premise open weights: compliance by architecture

Run the model where the data already is. As of September 2026 the open-weight bench is genuinely frontier-class: GLM-5.3-Flash (Zhipu AI's 320B-parameter MoE with 18B active parameters, MIT license, 1M-token context, native document and image understanding) handles summarization, drafting, and records digitization from a single node, with Qwen3.8 available where a second vision model is wanted. Weights are static files — checksummed, incapable of transmitting anything, air-gappable outright. PHI never crosses the network boundary, which means:

No business associate, no BAA — 164.502(e) never triggers.
No vendor breach surface — your Security Rule obligations (45 CFR 164.306–312: access control, audit controls, integrity, transmission security) apply to systems you already govern.
No cross-border or CLOUD Act analysis — nothing egresses, so there is nothing to compel from a provider.
Zero-marginal-cost batch work — overnight coding-suggestion queues, referral-letter drafting, and chart summarization run on electricity alone.

The honest trade-off: the covered entity owns the operational work — hardening, access control, monitoring, model upgrades. That is real work; it is also work hospital IT already does for the EHR, and it can be contracted as managed on-premise deployment without ever giving the operator custody of PHI.

What does the on-premise pattern look like in a clinic network?

The reference deployment we build for healthcare organizations: a rack inside the network (or air-gap-capable segment) serving a tuned language model for ambient scribing, referral letters, and coding suggestions, plus Qwen3-VL for faxes, forms, and scanned records; role-based access mapped to care teams; full audit logging for OCR-to-note lineage. Clinician-facing latency stays on-LAN, and the documentation queue drains overnight at zero marginal cost. For fully isolated segments — research enclaves, behavioral health — the same stack runs air-gapped; our air-gapped LLM deployment guide covers the telemetry and licensing gotchas.

How should a healthcare organization decide?

Volume and sensitivity decide it. Below roughly 2M tokens per day with genuinely de-identifiable workloads, architecture #2 or #3 can be rational. At clinical-workflow volume — ambient scribing alone generates millions of tokens per day across a modest clinic group — self-hosting is cheaper anyway (60–85% inference savings at sustained volume; see our on-premise cost breakdown) and removes the vendor from the risk register entirely. The question worth asking in 2026 is no longer "which vendor will sign our BAA?" It is "why is there a vendor in the PHI path at all?"