The Security Risks of AI Agents Are Already Here

AI agent risk in healthcare is not a future problem. It is already inside patient care, billing, scheduling, and documentation. If an agent can read PHI, call APIs, and change records or workflows, it can create security, privacy, patient-care, and HIPAA risk at the same time.

Here’s the short version:

• Prompt injection already works too often. Research cited in the article says jailbreaks worked in about 20% of cases, and about 90% of those led to sensitive data leaks.
• Healthcare agents often have too much access. That can turn one bad prompt into bulk PHI exposure, chart access across many sites, or mass record changes.
• The risk is not just data loss. Agents can also misroute messages, draft bad outputs, delay care, and disrupt billing or prior auth work.
• Human review is thin. One study in the article says clinicians spend about 11 seconds reviewing AI-assisted recommendations before accepting or changing them.
• Managing third-party AI risk is part of the problem. Model updates, hidden subprocessors, weak logging, and unclear PHI handling can leave health systems exposed.
• The fix starts with control basics. Use separate identities for each agent, least-privilege access, short-lived tokens, human approval for high-risk actions, tighter vendor review, and AI-specific incident response playbooks.

Put simply: if you treat an AI agent like a harmless assistant, you miss the risk. I’d treat it like a machine account with access to PHI, live systems, and patient-facing workflows.

A few numbers from the article make the point fast:

The core takeaway is simple: AI agent security belongs in current access control, vendor review, logging, and incident response work right now.

Why AI Agents Are Healthcare's Next Biggest Security Risk | Pranava Adduri

sbb-itb-535baee

Where AI Agents Create Security Exposure Today

AI agents create healthcare risk right now through prompt injection, broad access, weak integrations, and workflow tampering.

The first problem to watch is prompt injection. That’s when hidden instructions inside content an agent reads push it off course.

Prompt Injection and Unsafe Instruction Following

Prompt injection happens when hidden instructions inside content override the agent’s intended policy. Put simply, the agent reads something malicious and follows that instead of the rules it was supposed to follow.

OWASP's 2025 prompt-injection guidance explicitly lists poisoned documents, web pages, email content, attachments, and tool outputs as common indirect-injection sources^[2]. In healthcare, that can mean referral faxes, scanned outside lab reports, and patient portal messages.

A controlled study of medical LLMs found that prompt injections altered recommendations in 94.4% of 216 simulated patient–LLM dialogues and in 91.7% of high-harm scenarios, including unsafe pregnancy and drug-interaction recommendations, and concluded that current safeguards were insufficient to prevent clinically hazardous outputs^[1].

This gets worse fast when an agent already has broad access to tools and data. At that point, a bad instruction doesn’t just change a reply. It can trigger actions.

Excessive Access, Data Leakage, and Insecure Integrations

Even without an attacker, over-permissioned agents create serious exposure on their own. Many healthcare AI agents run with broad service account credentials or generic API keys that give them near-admin access to EHRs, data warehouses, HR systems, and productivity tools. When that happens, one bad prompt can lead to bulk data extraction, unauthorized chart access across multiple facilities, or mass demographic updates. That’s not a one-record mistake. It’s a system-level problem.

Agents that pull from shared drives, SharePoint, or cloud buckets can also surface PHI and internal policy text a user never requested. PHI may also spill into logs and telemetry sent to outside monitoring tools that weren’t built for HIPAA data. A healthcare privacy and compliance summary reported that 81% of all data policy violations in healthcare involved HIPAA-regulated data^[3]. That stat shows how easily PHI can move through channels no one meant to open.

Unapproved integrations make this even messier. A department team might connect a SaaS documentation assistant straight to the EHR with user credentials and skip vendor risk review altogether. On top of that, a vendor selling an AI scheduling assistant may rely on subcontracted LLM providers and outside data processors that never appear in the main contract. Those setups often lack data residency guarantees, clear PHI handling terms, incident notification commitments, and sub-processor transparency. Once PHI leaves the main system, the covered entity may have a hard time knowing where it went or what happened to it.

Workflow Manipulation and Operational Disruption

Agents that route messages, summarize charts, draft orders, or update downstream systems bring a different kind of risk: workflow integrity failure. This is the kind of issue that can slip by at first. Unlike a data breach, it may not set off an alert right away. It just sits there, quietly stacking errors until a clinician, patient, or auditor spots the damage.

The failure modes are pretty concrete. A care coordination agent might misread worsening chest pain in a portal message as routine follow-up and send it to a non-urgent queue. That kind of triage delay can affect patient safety directly. A compromised or malfunctioning agent handling inbound referrals or prior authorizations can also create backlogs, push up denial rates, and force staff into manual catch-up work. That hits revenue cycle performance, clinician workload, and patient throughput all at once.

Because these agents work at scale and move fast, one flawed prompt pattern or injected instruction can spread across hundreds or even thousands of records before anyone notices.

These are not just IT issues. They can turn into patient-safety, privacy, and compliance failures at the same time.

Why These Risks Matter for Cyber, Privacy, and Compliance Leaders

These risks create direct operational, privacy, and compliance exposure for CISOs, privacy officers, and compliance leaders. A technical failure doesn’t stay technical for long. Once it touches patient safety, audit trails, SOC 2 compliance, or breach response, it becomes a leadership issue fast.

Patient Safety and Care Delivery Consequences

Research shows that clinicians review AI-assisted recommendations for an average of only 11 seconds before accepting or overriding them ^[4]. That’s barely enough time to sanity-check a simple suggestion, let alone weigh a complex clinical recommendation. So when an agent produces an inaccurate output, that output can slide through a care workflow with very little scrutiny.

This hits harder in healthcare settings where AI is spreading at speed. As of mid-2026, the Department of Veterans Affairs had expanded its ambient AI documentation rollout from 10 facilities to more than 130 medical centers nationwide, handling sensitive veteran health data ^[4].

Over time, automation bias makes the problem worse. If staff start trusting AI by default, the human review step meant to catch mistakes can weaken. And once that safety backstop slips, errors have a much easier path to patients.

The same issue also turns into a compliance headache when PHI, prompts, and tool traces can’t be fully tracked or explained.

HIPAA, Auditability, and Breach Response Gaps

PHI exposure through AI agents can show up in logs, retrieval context, or tool-call traces. HIPAA’s audit control standard (45 CFR §164.312(b)) requires organizations to log and review activity in systems that handle PHI, including prompts, model versions, retrieval context, tool calls, and override decisions ^[4]. If that level of detail is missing, investigations get messy and response gets slower.

In March 2026, security researchers at Mindgard disclosed the "NEXUS" attack against Heidi Health's clinical scribe. The attack worked because the safety controls were stored in the prompt layer, not the model itself ^[4]. That detail matters. If incident response plans don’t cover prompt tampering, tool abuse, PHI spillover in logs, and large-scale behavior drift, then the plan has a hole in it.

That’s why AI agent controls need to sit inside access, governance, monitoring, and response programs now. AI agents need the same rigor applied to other high-risk systems, including:

• access control
• vendor oversight
• logging
• response planning

How Healthcare Organizations Can Reduce AI Agent Risk Now

AI agents are machine identities that read data, call tools, and trigger workflows. Security teams should govern them like privileged accounts. That changes how you handle identity, vendor access, and incident response.

Reducing AI agent risk starts with three controls: identity, vendor oversight, and incident response. Each one ties straight to the threats already on the table - prompt injection, excessive access, insecure integrations, and workflow manipulation.

Apply Least-Privilege Access and Strong Identity Controls

Every AI agent in your environment should have its own identity. Not a shared service account. Not a recycled admin credential. Treat each agent like a named identity: document who owns it, what it can do, which systems it can reach, and when that access ends. That inventory is the starting point.

From there, use a deny-by-default model. An agent should get only the permissions needed for its task, limited to the right data domain, and only for the time required. A chart-review assistant should read the current patient’s record during an active encounter - not run bulk queries across the entire EHR. A coding assistant can draft claims, but it should not be able to alter finalized billing records.

Short-lived credentials matter here. Ephemeral tokens, automatic rotation, and fast revocation cut down the damage window if an agent is manipulated or its credentials leak. For sensitive actions - modifying orders, exporting PHI, or sending messages at scale - add a human approval step before anything goes through.

Strengthen Vendor Due Diligence, Monitoring, and Governance

Your internal controls only go so far. If a vendor’s integration can still reach PHI or unapproved systems, the risk does not stop at your boundary. If a vendor’s agent touches PHI, that risk extends into the vendor’s architecture. Ask direct questions: how does the vendor block prompt injection from patient messages or free-text notes? Where is the model hosted? Is that environment covered by a Business Associate Agreement? What gets logged?

Retrieval sources need close review too. If an agent pulls in outside knowledge before answering, you need to know which knowledge bases or external corpora it can query and how PHI is protected during that process. Map subprocessors as well. If your vendor’s agent calls another vendor’s API or model, every link in that chain needs to meet your security and compliance bar.

The table below links each control to the risks it addresses:

Your organization also needs a clear list of approved use cases - such as pre-chart summarization, note drafting, and coding assistance - and prohibited actions - such as unreviewed order entry, bulk PHI export outside defined workflows, and connections to unapproved external APIs. A cross-functional AI governance committee should own that list and approve new deployments.

Extend Incident Response to Cover AI-Specific Scenarios

Even strong controls fail sometimes, so response plans need to cover agent-specific abuse. Standard incident response playbooks do not cover things like prompt tampering, tool abuse, or odd behavior across automated interactions. That gap needs to be closed before an incident hits.

When an AI incident is suspected, the first steps are clear: disable the agent’s tools or integrations, revoke or rotate its credentials, and isolate affected workflows. Then use the agent’s logs to piece together what happened - what prompts were sent, which tools were called, and what data was accessed - and decide whether PHI exposure meets HIPAA breach notification thresholds. Roles across security operations, privacy, compliance, clinical leadership, and vendor contacts should be defined ahead of time, including how to coordinate with third-party providers if their infrastructure is involved.

Each incident should feed back into tighter controls: narrower tool scopes, added approval steps, updated governance policies, and sharper monitoring rules. Run tabletop exercises for prompt injection, tool abuse, and rogue behavior before a real event.

Operationalizing AI Agent Risk Management with Censinet

Bring AI Agent Risk into Enterprise and Third-Party Risk Workflows

Controls only help if teams can run them the same way every time. Manual reviews start to fall apart when a health system is juggling dozens of AI-enabled workflows across clinical, admin, and revenue cycle work. At that point, the job shifts from having controls to making sure those controls work across every AI-enabled workflow.

Censinet RiskOps™ brings AI-enabled products, assessments, and remediation into one healthcare risk workflow. That means AI agent risks show up in the same dashboards and reports your board and audit committee already use for broader risk oversight, instead of living off to the side in a separate process.

That also means automated evidence collection, with human review built in. Censinet AI™ speeds up third-party vendor reviews by pulling out key integration details, like data flows from your EHR to a cloud-hosted LLM, and flagging fourth-party infrastructure that has not been reviewed. Censinet AI then sends findings to the right reviewers and keeps people involved where it matters most. It supports evidence validation, policy drafting, and risk routing, with human approval at every high-risk decision point.

Conclusion: Treat AI Agent Risk as a Present-Day Control Issue

AI agent risk needs to be handled inside the workflows healthcare teams already use. This is a present-day control issue, not something to put off. The practical move is to extend current access, governance, monitoring, and incident response workflows so they cover AI-enabled systems at scale.^[5][6]

FAQs

What counts as an AI agent in healthcare?

In healthcare, an AI agent is a system that can independently access, interpret, or act on Protected Health Information (PHI).

That’s what sets it apart from a standard data-processing tool. Instead of just handling data in the background, an AI agent can move through EHRs on its own, reply to patient messages, draft clinical notes, or send tasks to the right team.

And because these systems can shape clinical decisions or affect patient care workflows, the bar is the same as it is for human users. That means access controls, audit logs, and HIPAA compliance all still apply.

How can prompt injection affect patient care?

Prompt injection can put patient care at risk by overriding the intended logic of clinical AI systems. In a direct attack, someone can use a chat box or input field to get around safety rules.

Indirect injection is sneakier. Harmful instructions can be hidden inside clinical notes, PDFs, or referral letters that the AI pulls in. Once that happens, the system may return unsafe outputs or even trigger actions in the electronic health record that no one approved. The result can be diagnostic errors and poor care decisions.

What controls should we implement first?

Start with the basics first.

Build an enterprise-wide AI inventory that covers internal systems and third-party tools. Each tool should have a clear business owner and a clear security owner. If no one owns it, no one’s watching it.

Before any tool can access PHI, require review from security, privacy, legal, and clinical teams. That step matters. A tool might look harmless on the surface, but once patient data is involved, the risk changes fast.

Vendor checks need to go deeper too. Ask for an AIBOM, review data provenance, and put HIPAA-compliant BAAs in place that block unauthorized model training. In plain English: know what’s in the tool, know where its data came from, and make sure your contract says patient data can’t be used in ways you didn’t approve.

Then lock down access. Use least-privilege permissions so people only get what they need. Turn on phishing-resistant MFA, and keep audit logging in place so you can see who did what and when.

Related Blog Posts

• The Invisible Threat: How AI Amplifies Risk in Ways We Never Imagined
• The AI Cyber Risk Time Bomb: Why Your Security Team Isn't Ready
• AI Cyber Risk: When Your Smart Defense Becomes the Attack Vector
• Agentic AI Is Expanding Healthcare’s Attack Surface Faster Than Teams Realize