Shadow AI Sends Patient Data to Unapproved Tools

Patient Records Are Flowing Into Consumer AI Tools

A privacy officer cannot account for a disclosure that nobody recorded. That is the uncomfortable center of two incidents that surfaced in recent days, in which patient information reached AI tools that the organizations never approved, and where the people responsible for compliance learned about it after the fact.

An academic medical center disclosed that protected health information from a radiology research study had been entered into AI and machine learning tools in violation of its own policies. Separately, a staff member at a Washington State transitional care facility used a personal ChatGPT account to draft summaries of case notes containing names, diagnoses and medications. Neither case involved a classic attack. No one broke in. People with legitimate access to patient data moved it somewhere it should not have gone.

That distinction matters for how you defend against it. Perimeter tools were built to stop outsiders, and these incidents were insiders trying to work faster. This post covers what each incident involved, what HIPAA requires you to be able to show, where the real gap sits, and how to close it without taking productivity away from clinicians and researchers.

Key Takeaways

  1. The leak is an access problem, not an attack. Staff with valid credentials copied patient data into tools nobody approved, so controls aimed at outsiders did not see it.
  2. Two incidents show two patterns. One was a research team uploading imaging data, the other a staff member using a personal account to summarize case notes.
  3. Surveys suggest the behavior is common. Wolters Kluwer Health’s December 2025 survey found 40 percent of respondents had encountered unauthorized AI tools and nearly 20 percent admitted using them.
  4. The compliance burden is proof. Covered entities must be able to investigate, assess and document disclosures, which requires records of what left the environment.
  5. A governed channel beats a ban. Giving clinicians and researchers an approved path with logging and policy enforcement removes the reason to go around it.

What the Two Organizations Disclosed

The details come from reporting by The HIPAA Journal, which could not be retrieved directly for this post, so treat the specifics below as secondary reporting and confirm them against the organizations’ own notices before you cite them.

According to that reporting, the medical center discovered in September 2026 that protected health information used in a radiology research study had been entered into AI and machine learning tools, with uploads occurring in June 2026. The data reportedly excluded names, contact details, Social Security numbers and financial information, but included medical record numbers, radiology accession numbers, dates and times of service, facility and provider names, and clinical information that may have included diagnoses, treatment, surgical history and medications. The medical center reportedly found no misuse, suspended the research activity and asked for deletion. The number of affected individuals had not been disclosed.

In Washington State, the same reporting says a staff member at the Lake Burien Transitional Care Facility used a personal ChatGPT account to draft summaries of case notes. The state Department of Social and Health Services identified one instance, then multiple earlier ones, and wrote to 12 affected families. It also reportedly identified a similar incident involving four patients at Western State Hospital.

Notice what is missing from both accounts. Neither describes an enforcement action or a penalty. These are disclosure events that triggered investigation and notification, and that is exactly why they are useful to study. They show what ordinary, non-malicious AI use looks like when it goes wrong.

A Complete Checklist of HIPAA Compliance Requirements

Read Now

Why Medical Record Numbers and Dates Still Count

One detail in the medical center’s report is easy to dismiss. The data reportedly left out names and Social Security numbers. A reader might conclude the exposure was minor.

HIPAA does not work that way. The Privacy Rule defines protected health information broadly, and identifiers such as medical record numbers and dates of service are among the elements that make health data individually identifiable. De-identification under the Safe Harbor method requires removing a long list of identifiers, and that list includes medical record numbers and most date elements. A dataset that retains them is generally still PHI.

This is also where research teams get caught. Researchers handle data in ways that feel more like analysis than care, and they often assume that stripping direct identifiers is enough. The HIPAA Minimum Necessary Rule asks whether the amount of data used is limited to what the purpose requires. Sending identifiable clinical detail to an external tool for a task that did not need it is the kind of decision that rule is meant to prevent.

The Shadow AI Numbers Behind the Incidents

If these were isolated mistakes, they would be a training problem. The survey evidence suggests they are closer to a pattern.

Wolters Kluwer Health commissioned a survey of 518 healthcare professionals, split evenly between providers and administrators, fielded online in December 2025 by CITE Research. In the company’s release, 40 percent of respondents said they had encountered an unauthorized AI tool in their organizations, nearly 20 percent said they had used one, and about one in 10 reported using an unauthorized tool for a direct patient care use case. Note the wording. The survey measures encounter and admission, not volume of PHI exposed, and it comes from a vendor with a stake in the market.

A second data point comes from outside healthcare. The National Cybersecurity Alliance and CybSafe’s Oh, Behave! research reported that 32 percent of workplace AI users had shared sensitive company data with AI tools without their employer knowing. Different population, same behavior.

Read these as directional. They do not tell you your exposure. They tell you that the assumption “our staff are not doing this” is hard to defend, and that an organization claiming zero shadow AI use probably lacks the visibility to know.

What HIPAA Expects You to Be Able to Show

Start with what the rule does and does not say. HIPAA has no AI specific provision. The rules that matter are the existing ones, applied to a new tool.

When PHI is disclosed to a party without a valid business associate agreement or other permission, that is generally an impermissible disclosure. The breach notification framework then requires a risk assessment to decide whether the incident is a reportable breach, using factors such as the nature of the information, who received it, whether it was acquired or viewed, and how far the risk has been mitigated. You must be able to run that assessment with real facts.

That is where records matter. To assess an AI related disclosure, a privacy officer needs to know which patient data went to which tool, when, by whom, and whether the recipient retained it. A consumer AI account on a personal device generates none of that on your side. You learn of it, if at all, from a whistleblower, a state agency or an audit.

The accounting of disclosures requirement points the same direction. Individuals have a right to request an accounting of certain disclosures of their PHI, and covered entities must be able to produce it. An unrecorded paste into a chatbot is a disclosure that your accounting process cannot see. Whether a given AI disclosure falls inside the accounting requirement depends on the facts and the exceptions, so involve counsel. The operational point holds either way. You cannot answer a question about something your systems never recorded.

On the broader security side, the HIPAA Security Rule requires audit controls and access management. The Security Rule overhaul that HHS proposed in early 2025 is still a proposed rule, not a final one, and today encryption remains an addressable specification, so avoid telling your board that new requirements are already in force. The existing audit control standard is enough to make the argument. If you cannot examine activity involving electronic PHI, you have a gap.

Why Blocking Consumer AI Does Not Close the Gap

The reflex response is to block ChatGPT and its peers at the network edge. It helps at the margin and fails as a strategy, for three reasons.

Staff use personal devices and personal accounts, as the Lake Burien case shows. A network block does nothing about a phone on a cellular connection.

Second, the tools keep multiplying. Every productivity application now embeds an assistant. A block list written today is stale next month.

Third, the pressure that drives the behavior remains. A clinician drafting case summaries at the end of a long shift has a real problem, and the tool solves it. Remove the tool without offering a replacement and people find another route. Dismissing the motive is how shadow AI grows.

The more durable approach is to meet the need inside a governed environment. If a research team wants to use machine learning on radiology data, give them a channel where the data stays under policy, access is limited to what the project requires, and every operation is logged. If a clinician wants a draft summary, provide an approved assistant that reaches patient content only through controlled access.

What a Governed Path Looks Like

A governed AI path for patient data has a handful of properties, and each maps to a compliance question.

Identity. The AI acts on behalf of an authenticated user, and its permissions never exceed that user’s. This answers the question of who authorized the access.

Policy at the point of access. Each request is evaluated against rules based on the data’s sensitivity, the user’s role and the context. This supports the minimum necessary standard and keeps a research assistant out of records outside its protocol.

Complete logging. Every AI operation is recorded with enough detail to reconstruct it later. This is what lets a privacy officer perform the breach risk assessment and respond to a patient’s accounting request with facts.

No credentials in the model. Authentication tokens stay outside anything the AI can read, which closes off a class of prompt injection risk.

Kiteworks designs its Secure MCP Server and Kiteworks Compliant AI around these properties, so that AI assistants reach content in the Kiteworks secure data exchange under the same role and attribute based policies and the same consolidated audit log that govern human users. The honest limit is scope. Kiteworks was not in the data path of either incident, and it cannot stop a staff member from pasting text into a personal account on a personal phone. The claim is that when patient content is accessed through Kiteworks, the access is governed and recorded. That is the evidence a privacy officer needs for the content that matters most, and it gives you something concrete to point to when staff ask what they are allowed to use. For the supporting framework, see AI data protection guidance and HIPAA compliance.

A 30 Day Plan for Privacy and Security Officers

You do not need a program rewrite to make progress. Four moves fit a month.

In week one, ask your clinical, research and administrative leaders a plain question. Which AI tools are you using for work, including personal ones? Promise no discipline for honest answers. The point is to learn the real picture, and amnesty gets better data than enforcement.

In week two, classify the use cases. Separate tasks that never involve PHI from those that do. For the PHI cases, decide which can be served by an approved tool and which should stop.

In week three, stand up one governed pathway for the highest volume use case. Route it through a channel that enforces policy and logs activity. Prove the logging works by pulling a record of a test operation.

In week four, update your breach risk assessment procedure and your accounting of disclosures process to include AI related events, and test them with a scenario. Time how long it takes to find out what was shared.

Add a short, concrete staff message that explains why personal AI accounts are off limits for patient data and what to use instead. People follow rules they understand, particularly when an alternative exists. Pair this with PII and PHI handling guidance so staff know what counts.

What to Take to Your Next Compliance Committee

Frame the issue as a records problem with a clear owner. The organization already has duties to investigate, assess and document disclosures. AI adds a new route by which PHI leaves, and your current records do not capture it.

Bring three facts. These incidents were not attacks. The survey evidence suggests the behavior is widespread. And the remedy is a governed alternative with logging, not a prohibition. Then ask for a decision on which governed pathway to build first, and who owns the weekly review of its logs.

The organizations that handle this well will not be those with the strictest policies. They will be the ones that can answer, quickly and with evidence, the question every regulator and every affected patient eventually asks. What happened to my data, and how do you know?

Research Data Deserves Its Own Controls

The academic medical center case involved a research study, and research environments create risks that clinical operations do not. Researchers move data between systems, apply new analytical tools and collaborate across institutions. They value speed and flexibility, and they sometimes treat identifiable data as a working material rather than as regulated information.

Machine learning makes this sharper. A researcher exploring a model on imaging data may reach for a hosted tool because the local environment lacks the compute. The upload feels like a technical step. In regulatory terms it is a disclosure to a third party. Institutions that run research programs need explicit rules about where identifiable and limited data sets may go, who approves exceptions and how those approvals get recorded.

A few controls help. Maintain an approved list of analysis environments, each with the right agreements and security reviews. Require data use agreements that spell out permitted tools. Provide a governed workspace where researchers can run analysis without moving data out, so the convenient option is also the compliant one. And log access to research data sets at the level of the individual operation, so that if an upload occurs you can see it.

Institutional review boards and privacy officers should also talk to each other. Study protocols often describe data handling in general terms. Adding a short section on AI and machine learning tools, with named approved options, turns a vague expectation into a checkable commitment.

Business Associate Agreements and Vendor Due Diligence

Many organizations assume that signing up for an enterprise AI plan solves the HIPAA question. Sometimes it does, and sometimes it does not, and the difference lives in the paper.

A business associate agreement is the contract that allows a vendor to handle PHI on a covered entity’s behalf. Without one, disclosing PHI to the vendor is generally impermissible. With one, the vendor accepts defined obligations around safeguards, use limits and breach reporting. Whether a particular AI product is covered depends on the vendor’s offering, the plan and the configuration, so read the terms rather than the marketing.

Ask vendors specific questions. Will the agreement cover this product and this plan? Is customer data used to train models, and can that be turned off contractually and technically? Where is data stored and for how long? Can you obtain logs of what your users submitted? What happens to data on termination? What is the vendor’s breach notification window?

Then verify the answers on the product. A contract that promises no training on your data matters only if the settings match. Test with non-sensitive data, confirm retention behavior and document the result.

Finally, remember that approved vendors cover only the paths you know about. The Lake Burien example involved a personal account, which no agreement touched. Vendor due diligence and shadow AI controls are two halves of the same program. For the underlying data classification work, see the Kiteworks overview of PII and PHI and its AI data governance guide.

What to Log When AI Touches Patient Data

Logging is easy to promise and easy to get wrong. A log that records only that a user logged in tells a privacy officer almost nothing. A useful record of AI access to patient content captures several things.

The identity of the person on whose behalf the AI acted, and the identity of the AI system itself. The specific content accessed, down to the file or record. The action taken, such as read, summarize, copy or download. The time. The policy that was evaluated and its outcome, including denials. And the destination, where the data went next.

Retention matters as much as content. Breach investigations can begin long after the event, as the academic medical center’s timeline shows, with discovery in September of activity that occurred in June. Logs that roll off after 30 days would not support that investigation. Set retention to match your regulatory and investigative needs, and protect logs from alteration.

Make the logs usable. Feed them to your SIEM so alerts fire on unusual patterns such as bulk access to patient records by an AI account. And make sure compliance staff, not only security engineers, can query them. A privacy officer who must file a ticket and wait a week for an answer cannot meet a notification clock.

Training That Changes Behavior

Annual compliance training rarely changes how people use AI tools. What works is specific, practical and timed to the moment of decision.

Use real examples from your own sector. The medical center and Lake Burien cases are relatable because they involve ordinary tasks, summarizing notes and exploring a model. Show staff that the people involved were not villains, and that the data they entered carried identifiers that mattered.

Offer the alternative in the same breath as the prohibition. Staff who learn only what they cannot do will find workarounds. Staff who learn that an approved assistant exists, what it can do and how to request access are more likely to use it.

Make reporting safe. People who accidentally paste PHI into a consumer tool should be able to say so quickly, without fear of automatic punishment. Early reports shorten the breach assessment and improve mitigation, since you can ask for deletion while it still helps.

Reinforce with reminders where the work happens. A short banner in the approved assistant, a note on the research data portal, or a line in the clinical documentation template can do more than a slide deck. Measure the result with periodic surveys and with the logs from your governed channel, and adjust.

To see how governed AI access to sensitive content looks in practice, request a Kiteworks demo and review the audit trail with our team.

Frequently Asked Questions

It can be. Without a business associate agreement or other permission, disclosing PHI to a consumer AI service is generally an impermissible disclosure. Enterprise offerings with a signed agreement are treated differently. Check the vendor’s terms and involve counsel. Review HIPAA compliance requirements for the baseline.

Data can remain PHI when it keeps other identifiers, such as medical record numbers, accession numbers or dates of service. De-identification requires removing a defined list of identifiers or an expert determination. Researchers often miss this. See the PII and PHI overview.

Combine an honest staff survey with technical signals such as network and endpoint telemetry, and compare the results. Surveys surface personal device use that tools miss. An amnesty approach improves candor. Then route approved use through logged channels to build a complete audit trail.

It depends on a documented breach risk assessment covering the data involved, the recipient, whether it was acquired or viewed, and mitigation. Some incidents are reportable and some are not. State laws may add duties. Document your reasoning either way and consult counsel.

Provide an approved assistant that reaches patient content only through controlled access, with role and attribute based policies and full logging. Pair it with training on why personal accounts are off limits. Explore a governed AI access model built for this.

Additional Resources

Get started.

It’s easy to start ensuring regulatory compliance and effectively managing risk with Kiteworks. Join the thousands of organizations who are confident in how they exchange private data between people, machines, and systems. Get started today.

Share
Tweet
Share
Explore Kiteworks