Yes, you can use AI with protected health information, but only when three conditions hold: a signed Business Associate Agreement (BAA) that actually covers the workflow in question, technical safeguards including encryption, operation-level access control, and tamper-evident audit logs, and documented deployment controls tied to a formal risk analysis. A vendor’s marketing page calling itself “HIPAA compliant AI” is not one of those three conditions. Evidence is.
TL;DR:
- Using AI with protected health information requires a signed Business Associate Agreement that explicitly covers the exact workflow, not just a generic compliance claim.
- Vendors must provide current SOC 2 Type II reports, clear BAA scope matching your product features, and a detailed data flow diagram to verify HIPAA compliance.
- Technical safeguards like operation-level access control, encryption, customer-controlled keys, and tamper-evident audit logs are essential for HIPAA security adherence.
- AI output containing identifiable patient data is considered protected health information unless it has been properly de-identified according to HIPAA standards.
- Building an AI system with full ownership of architecture and data flow ensures better compliance and scalability compared to relying on vendor platforms with limited scope.
Table of Contents
- What Makes AI HIPAA Compliant: Business Associate Status and BAA Scope
- Vendor Evidence and the Questions to Ask Before You Sign
- Which Technical Controls Actually Satisfy the HIPAA Security Rule?
- How Do You Apply Minimum Necessary in AI Prompts and Training Data?
- What Belongs in Your AI Risk Analysis and Incident Response Plan?
- Does GDPR Change How You Deploy AI With Patient Data?
- How Autonomousfirm Approaches Compliance-First AI Builds
- Who Needs to Be Trained Before an AI Deployment Goes Live?
- What Data Retention and Deletion Rules Apply to AI Models Trained on PHI?
- Is AI Output Considered PHI Under HIPAA?
- Choosing Between a Vetted Vendor and a Custom Build
- Ready to Build a Compliant AI System You Actually Own?
- Sources
- FAQ
What Makes AI HIPAA Compliant: Business Associate Status and BAA Scope
HIPAA applies the moment a vendor creates, receives, maintains, or transmits protected health information (PHI) on behalf of a covered entity. That is the entire test, according to HHS guidance on business associates. If an AI tool touches PHI in any of those four ways, it is a business associate, full stop, and it needs a signed BAA before a single patient record enters the system.
The BAA itself has to name the actual workflow you’re running, not just exist as a boilerplate document sitting in a vendor’s legal folder. A BAA that was drafted for basic file storage does not automatically extend to a chatbot summarizing clinical notes or an agent pulling data from an EHR.
A workable BAA needs to spell out:
- Permitted and prohibited uses of PHI, specific to the actual product features you’re deploying
- Breach notification obligations, including exact timelines and the method of notice
- Subcontractor and subprocessor coverage, meaning every downstream vendor that might touch the data
- Data return or deletion terms once the engagement ends or a patient exercises a deletion right
The most common failure point is scope. A vendor signs a BAA for its “enterprise tier,” then the clinical team quietly starts using a consumer-facing feature or a beta tool that was never named in that agreement. Unnamed subprocessors are the second recurring gap. Cloud AI stacks often route requests through several third-party services, and if one of them isn’t listed in the BAA chain, you have PHI sitting outside your contractual protection without anyone realizing it.
Vendor Evidence and the Questions to Ask Before You Sign
A vendor telling you their product is HIPAA compliant AI is a marketing claim, not proof. Proof looks like documents you can actually read, dated reports you can verify, and a live demonstration of controls working in front of you.
Ask for these in order:
- A current SOC 2 Type II report, not a Type I. Type I checks whether controls exist on a given date; Type II checks whether they actually operated correctly over a period of months. Confirm the report’s date range covers recent activity, not a report from two years ago that nobody has refreshed.
- Independent HIPAA audit documentation, if the vendor has commissioned one, along with any FedRAMP authorization if the deployment touches federal data.
- The exact BAA scope, cross-checked against the specific product tier, API, or feature set you plan to use. This is where OpenAI’s own documentation on HIPAA-eligible products is useful as a model. It explicitly states that eligibility depends on configuration, meaning not every feature under a given brand is automatically covered.
- A subprocessor list, current and named, not a generic statement that “industry-standard partners” are used.
- A data flow diagram showing where PHI physically travels, where it’s stored, and where it’s deleted.
- A live or recorded walkthrough of audit logs, so you can see whether the system actually records who did what, or just claims to.
Pro Tip: Ask the vendor to show you an audit log entry for a real, specific action, not a screenshot from a slide deck. If they can’t produce one on the spot, the logging capability probably isn’t as mature as the sales page suggests.
Peer-reviewed analysis of AI chatbots in clinical settings has repeatedly flagged the same gaps: thin audit trails, access controls that stop at the folder level instead of the individual action, and a wide gulf between what vendors claim and what deployment evidence actually shows. Treat every vendor claim as a hypothesis to test, not a fact to accept.
Which Technical Controls Actually Satisfy the HIPAA Security Rule?
Encryption is the baseline everyone gets right and almost nobody stops at, which is the actual problem. The HIPAA Security Rule lists encryption as an “addressable” implementation specification, meaning covered entities must assess the risk and document their decision rather than being handed a fixed mandate, according to HHS’s Security Rule guidance. In practice, strong encryption protocols at rest and in transit are the working standard for anything touching PHI, but the harder question is where the encryption boundary actually ends. Many AI vendors encrypt data in their own infrastructure, then decrypt it the moment it hits a third-party inference API. If you don’t know exactly where that boundary sits, you don’t actually know your encryption posture.
Key management matters as much as the algorithm. Customer-controlled keys, where the healthcare organization holds the ability to revoke access independent of the vendor, give you a kill switch that vendor-managed keys don’t.
Access control is where most deployments quietly fail. Coarse, folder-level permissions (“this team can access this drive”) look reasonable on paper but don’t map to how AI agents actually work. An agent might need to read a lab result to answer one question and have no business touching a psychiatric note five minutes later. That’s what attribute-based access control (ABAC) is for: permissions evaluated at the level of the specific operation, not the folder. Guidance from European data protection regulators has pointed to operation-level access control paired with tamper-evident logging as the practical evidence that actually holds up in review, far more than a broad access policy document.
A short list of what a defensible technical stack includes:
- Encryption in transit and at rest, with a documented risk-based decision, not an assumption
- Customer-controlled or customer-revocable keys, not vendor-only key custody
- ABAC enforced at the operation level, not the folder or workspace level
- Tamper-evident audit logs that record which human authorized each specific AI action, not just system-level uptime logs
One overlooked factor: retention settings change your BAA coverage entirely. OpenAI’s own documentation notes that certain API configurations require modified retention settings and a designated regulated workspace before HIPAA eligibility even applies. Default retention on a standard tier is often explicitly excluded from the BAA, which means an engineer flipping a setting back to default can silently move an entire workflow outside compliant territory.
How Do You Apply Minimum Necessary in AI Prompts and Training Data?
The “minimum necessary” principle means giving an AI system access only to the PHI required for the specific task. That is a different problem from de-identification, and healthcare teams routinely conflate the two.
De-identification strips or generalizes identifiers so the data no longer counts as PHI at all. Minimization keeps the data identifiable but limits exposure during any single operation. A clinical summarization tool might need a patient’s full chart to write a useful summary, so de-identification isn’t the answer there. Minimization is: give the model exactly the fields the task needs and nothing else.
Practical techniques that work in production:
- Field-level prompt construction, where the application layer assembles a prompt from only the specific fields a task requires, instead of dumping an entire record into context.
- Synthetic data for model training or fine-tuning, so the model never touches real patient records during the learning phase.
- Redaction pipelines that strip identifiers before data reaches a general-purpose model, reserving full PHI access for narrowly scoped, audited workflows.
- Rate and threshold controls that flag or block unusual volumes of PHI extraction from a single account or agent, catching both misuse and compromised credentials.
Pro Tip: Log every field an AI system requests, not just what it returns. A sudden pattern of requests for fields unrelated to the stated clinical task is often the first sign of scope creep or a compromised integration, long before any data actually leaks.
None of this eliminates risk entirely. Large language models can memorize training examples, and a model trained even briefly on real PHI carries some re-identification risk that redaction after the fact can’t fully undo. That’s the strongest argument for keeping real PHI out of training pipelines altogether and reserving it for tightly scoped, logged inference operations instead.
What Belongs in Your AI Risk Analysis and Incident Response Plan?
A risk analysis for an AI system needs to function as a real document a regulator could review, not a checkbox exercise. For every distinct AI use case touching PHI, document:
- The specific purpose of the AI operation, described precisely enough that “helps with documentation” isn’t an acceptable answer.
- The exact categories of PHI involved, down to specific field types rather than a general reference to “patient data.”
- Every recipient, human and system, including any subprocessor that touches the data downstream of the primary vendor.
- The retention period, tied to the actual configuration in use, not the vendor’s default marketing description.
Not every AI deployment needs a formal Data Protection Impact Assessment (DPIA) under a US-only HIPAA framework, but any organization processing EU patient data, or any deployment involving large-scale automated decision-making, should treat a DPIA-equivalent process as standard practice. Legal or a designated privacy officer should sign off before go-live, not after a pilot has already been running with real patient data for three months.
Breach notification timelines belong in the contract language itself, not just in a general HIPAA policy binder. If your BAA doesn’t specify how quickly a business associate must notify you of a suspected breach, and by what method, that gap becomes your problem the day something actually goes wrong.
Does GDPR Change How You Deploy AI With Patient Data?
Organizations handling both US and EU patient data can’t treat HIPAA as a complete compliance framework, because GDPR imposes its own separate obligations on top. The European Data Protection Board’s guidance on AI models requires a case-by-case assessment of whether data is genuinely anonymous, rather than accepting a vendor’s blanket anonymization claim, and it expects documented reasoning behind any legitimate-interest basis for processing.
That legitimate-interest test runs in three steps: identifying a genuine, specific interest; confirming the processing is actually necessary for that interest; and balancing it against the individual’s rights. Skipping straight to “we have a legitimate interest” without working through all three steps is exactly the kind of shortcut EU regulators flag.
GDPR compliance also tends to demand more documentation than a HIPAA-only program requires, including DPIAs for higher-risk processing and explicit transfer mechanisms when data crosses borders. A model trained partly in one jurisdiction and deployed in another needs a documented legal basis for that transfer, not an assumption that cloud infrastructure makes borders irrelevant. Industry guidance is blunt on this point: a vendor’s SOC 2 or ISO certificate does not substitute for the controller’s own documentation of purpose, lawful basis, and minimization. That responsibility stays with the healthcare organization, not the vendor.
For multinational teams, the practical action list is short: document the lawful basis for every AI use case touching EU data, run a DPIA where the processing is high risk or large scale, and confirm your data transfer mechanism and storage location before a single record moves.
How Autonomousfirm Approaches Compliance-First AI Builds
Most of the failure points covered so far—unnamed subprocessors, folder-level access instead of operation-level control, retention settings nobody rechecked—share a common root cause: the organization is renting someone else’s system and inheriting whatever compliance gaps that system already has.
Certain providers build AI-native systems designed for regulated industries such as healthcare and finance, emphasizing client ownership of the platform instead of licensing. That ownership model matters directly to everything in this article: when you own the system, you control the BAA scope, the retention configuration, and the audit logging design from day one, instead of negotiating around a vendor’s existing product boundaries.
The approach centers on:
- Building compliance, security, and data sovereignty into the architecture itself, so audit trails and access controls aren’t bolted on after a pilot succeeds
- Transferring the proprietary knowledge and technical control needed for a client’s team to actually run and govern the system going forward
- Structuring partnerships so a healthcare organization can scale automated workflows without a proportional increase in headcount
That last point tends to matter most for clinical operations teams stretched thin already: automating a documentation or intake workflow shouldn’t require hiring an entire new department just to keep it compliant.
Who Needs to Be Trained Before an AI Deployment Goes Live?
Technical controls fail quietly when the humans operating around them don’t understand what those controls actually do. Staff training for AI systems handling PHI needs to cover more ground than a standard annual HIPAA refresher.
Clinical and administrative staff using an AI tool day to day need specific training on what counts as an appropriate prompt. Pasting a full patient chart into a general-purpose chatbot because it’s faster than the sanctioned tool is one of the most common real-world violations, and it usually happens because nobody explained why the sanctioned tool exists in the first place.
IT and security teams need a different layer of training entirely, focused on configuration drift. A retention setting or access policy that was correct at deployment can silently change during a routine platform update, and someone needs to own a recurring check that reconfirms the compliant settings are still active.
Update protocols matter just as much as initial training. Every vendor product update, every model version change, and every new feature release needs a review step before it reaches production, because a feature that was outside BAA scope at launch can be added to a tier without anyone renegotiating the contract. Build a recurring calendar review, quarterly at minimum, that revalidates BAA scope against the current feature set actually in use, rather than assuming the agreement signed eighteen months ago still matches reality.
What Data Retention and Deletion Rules Apply to AI Models Trained on PHI?
Retention policy for an AI system touching PHI has two separate layers that get confused constantly: retention of the raw data passing through the system, and retention of anything the model itself may have learned from that data during training or fine-tuning.
Raw data retention is the more straightforward layer. Your BAA should specify exactly how long inputs, outputs, and logs are stored, and confirm that deletion requests are honored on a defined timeline rather than “eventually.” Modified retention settings, the kind referenced in OpenAI’s HIPAA-eligible product documentation, exist precisely because default retention periods on standard tiers are often too long, or too vague, for compliant use.
Model-level retention is harder. If a model was fine-tuned on real PHI rather than synthetic or de-identified data, deleting the source records doesn’t necessarily remove what the model learned from them. This is why the safest architecture treats production models as fixed and PHI-free at the training layer, reserving real patient data for retrieval-based systems where the record can be deleted from a database outright, rather than baked into model weights that are difficult to unlearn.
If retraining on real PHI is genuinely necessary for clinical accuracy, that decision needs its own documented risk analysis, a defined retraining cadence, and a clear position on what happens to the previous model version once retraining completes. Deleting a database record while an old, still-deployed model version retains the pattern it learned from that record is not actually deletion.
Is AI Output Considered PHI Under HIPAA?
An AI system’s output is PHI whenever it’s derived from or contains identifiable patient information, regardless of whether the output looks like a fresh piece of text the model generated. A summary of a patient’s lab history is PHI even though the model “wrote” it, because the underlying content is still tied to an identifiable individual.
This distinction trips people up constantly with generative outputs. A clinician might assume that because a model synthesized new sentences rather than copying a record verbatim, the output somehow escapes PHI status. It doesn’t. The test is whether the output allows a reasonable person to identify or infer information about a specific patient, not whether the wording is original.
The compliance implication is direct: any storage, transmission, or downstream use of AI-generated output touching patient information triggers the same Security Rule obligations, encryption, access control, audit logging, as the source data itself. An output log that isn’t encrypted, or a downstream analytics tool ingesting AI-generated clinical summaries without its own BAA, creates exposure just as real as a leak of the original chart.
Output genuinely stops being PHI only when it has been properly de-identified according to HIPAA’s Safe Harbor or Expert Determination methods, or when it was generated from data that never touched a real, identifiable patient in the first place, as with a system trained and operated entirely on synthetic records. Short of that, treat every AI output touching patient context as PHI by default, and require your compliance team to prove otherwise rather than assume it.
Choosing Between a Vetted Vendor and a Custom Build
A vetted vendor with clean BAA scope and current audit evidence is the right call when your workflow is standard, speed matters, and the PHI surface area is narrow, think routine documentation support or scheduling assistance. Trust the paper trail described earlier: SOC 2 Type II, named subprocessors, a BAA that actually covers the feature you’re using.
A build or technical partnership becomes the stronger option once your PHI surface area grows, your workflow is genuinely custom, or you need audit granularity a generic vendor product wasn’t designed to provide. Data sovereignty concerns, high regulatory exposure, or a strategic need to own the resulting IP all point the same direction. Ask yourself how much PHI actually flows through this system, and how much control over that flow you need to sleep at night. That answer usually settles the vendor versus build question faster than any feature comparison could.
— Matevz
Ready to Build a Compliant AI System You Actually Own?
Vetted vendors work fine for narrow, standard workflows, but once your PHI surface area grows or your compliance team needs audit granularity a generic product wasn’t built to provide, renting someone else’s platform starts costing more than it saves. Autonomousfirm builds AI-native systems for regulated industries where the client owns the architecture, the data flows, and the audit trail from day one, instead of hoping a vendor’s BAA scope happens to match the workflow you actually need.
That ownership model is what lets healthcare organizations scale automated workflows without scaling headcount at the same rate, since the compliance and access controls are built into the system rather than negotiated around it after the fact. If your team is weighing a custom build against another vendor contract, request an evaluation of your deployment and get a clear read on what a compliance-first architecture would actually look like for your workflow.
Sources
- Business associates | HHS
- EDPB opinion on AI models: GDPR principles support responsible AI
- AI Chatbots and Challenges of HIPAA Compliance for AI Developers and Vendors (PMC)
FAQ
Does ChatGPT Have a HIPAA-Compliant Version?
OpenAI offers specific HIPAA-eligible products and configurations under a signed BAA, but eligibility depends on settings like modified retention and a regulated workspace, so not every ChatGPT feature or tier qualifies automatically.
Is AI Prohibited by HIPAA?
No. HIPAA doesn’t ban AI outright; it requires that any AI vendor touching PHI sign a BAA and meet Security Rule safeguards, meaning compliance is a matter of deployment configuration rather than a blanket restriction.
What Is the 30% Rule in AI?
There is no recognized “30% rule” under HIPAA, GDPR, or established AI governance frameworks; if you’ve seen the term elsewhere, it isn’t a standard compliance benchmark and shouldn’t be treated as one.
Does OpenAI Have a HIPAA-Compliant Version?
OpenAI provides HIPAA-eligible products through its enterprise and API offerings under a signed BAA, with eligibility tied to specific configurations rather than applying uniformly across every OpenAI product.
