Agentic AI Security

The AI Act Now Recognises Agents. Its Security Rules Still Miss Them.

The European Commission’s AI Act Service Desk answers the question directly. AI agents are not a separate category of AI under the AI Act, and the definitions of an AI system in Article 3(1) and of a general-purpose AI (GPAI) model in Article 3(63) are sufficient to cover them. For the models underlying agents, the Commission names autonomy and tool use as factors that can be decisive in designating a model as presenting systemic risk under Article 51(1)(b) and Annex XIII, point (e). It then points to the risk management obligations owed by providers of such models, operationalised through measures in the Safety and Security chapter of the GPAI Code of Practice.

The same page says the Commission’s regulatory considerations on agents are only preliminary at this stage.

Every route the Commission names operates at the GPAI model layer, through systemic-risk designation and a voluntary code that around two dozen frontier providers have signed. None of it reaches Chapter III, which contains the security requirements for high-risk AI systems and which governs the conformity assessments that organisations deploying agents into hiring, credit, healthcare and critical infrastructure will conduct from December 2027.

Here is the sentence for your CISO. The Act covers your agent, the Commission says so, and the provision that will govern its security assessment enumerates attacks on training data and model inputs while naming none of the four ways agents actually fail. Compliance will not surface those risks, so your own threat modelling has to.

What Article 15 names, and what it leaves to you

Article 15(5) requires technical solutions appropriate to the relevant circumstances and the risks. It then provides that those solutions must include, where appropriate, measures addressing manipulation of the training data set, manipulation of pre-trained components, inputs designed to cause a mistake, confidentiality attacks and model flaws.

That list is illustrative, and the Act says so. The where appropriate qualifier and the model flaws catch-all leave room for anything else a careful provider chooses to address. The difficulty is that every named item targets a fixed point in a training or inference pipeline, so a provider working from the list as a checklist will produce a file that misses an agent’s real exposure entirely.

Four failure modes go unnamed.

Indirect prompt injection. An agent retrieves a web page, a document or an email containing text the model treats as instruction. Neither the training data nor the pre-trained components were manipulated, and the user's input was benign. The attack arrives through content the agent fetched while doing what it was told. securing.ai/ covers the mechanism in the context of semantic adversarial attacks. Injection is untrusted content becoming instruction. Jailbreaking is a user subverting the model’s own policy. The control that stops one does not stop the other.

Persistent memory corruption. Retrieval from an agent’s store is functionally input, so Article 15 is broad enough to reach it. What the provision’s input-evasion framing does not capture is lifecycle. A single successful injection written into an agent’s persistent store changes behaviour on every later task, long after the request that planted it has finished. Point-in-time input validation does not address state that outlives the request.

Misuse of delegated tool authority. The agent uses a legitimately granted permission to do something the operator did not intend. No component was compromised. The model produced a plan and the plan executed with valid credentials. Article 15 does not identify misuse of delegated authority as a distinct attack class, which leaves providers to translate a model-security provision into an authorisation problem at system level.

Compounding across a trajectory. Repeated autonomous actions create opportunities for errors and attacks to accumulate across a task. Whether that accumulation matters depends on the per-step error rate, the correlation between steps and whether the agent can recover, and none of those is addressed by a provision written for systems that return one output for one input.

The OWASP GenAI Security Project released the OWASP Top 10 for Agentic Applications on 9 December 2025, ten categories numbered ASI01 to ASI10, and it covers this territory. The Act does not reference it and has no equivalent taxonomy.

The role allocation strains at organisational boundaries

The Act allocates obligations between providers, who develop a system and place it on the market, and deployers, who use it under their own authority. Article 25 sets out when a deployer or third party becomes a provider of a high-risk system: putting its name or trademark on the system, making a substantial modification to a system already classified high-risk, or changing the intended purpose of a system so that it becomes high-risk.

Organisation A builds an agent framework. Organisation B configures it with a system prompt, a set of tools and credentials, and deploys it against its own data. Organisation C supplies the underlying model through an API. Organisation D operates a service the agent calls as one of its tools.

B’s position depends on facts. Configuring a framework with prompts, tools and credentials does not by itself meet any of the Article 25 triggers. Whether B has made a substantial modification within Article 3(23), or changed an intended purpose, is a question about what B actually did. Providers should not assume either answer.

D is the harder case. D’s service is used by an agent that selected it at runtime, possibly without D knowing it is serving an agent and not a person. Article 25(4), in the Act as originally adopted rather than as amended, requires the provider of a high-risk system and the third party supplying tools, services, components or processes integrated into it to specify by written agreement the information, capabilities, technical access and other assistance the provider needs in order to comply. The paragraph exempts third parties releasing those tools publicly under a free and open-source licence, other than general-purpose AI models. A written agreement presumes the provider knows which third party it is dealing with before the system is assessed. An agent selecting a service at runtime does not produce that relationship, and the Digital Omnibus left the provision alone.

Substantial modification raises the same problem across time. Conformity assessment happens once and produces a static artefact. An agent that acquires a new tool on Tuesday has a capability surface and a blast radius the Monday assessment did not cover. No guidance says where configuration ends and substantial modification begins for agentic systems.

Human oversight assumes a reviewable moment

Article 14 requires high-risk systems to be designed so natural persons can oversee them effectively. The provision is more capable than critics allow: it contemplates monitoring operation, disregarding, overriding or reversing outputs, and intervening or stopping the system.

Every one of those controls assumes the effect can still be withdrawn. Overriding an output works where the output is a recommendation. Reversing works only where the action can be undone. An agent that has sent the email, updated the record and called the payment API has produced external effects that no override restores. For those actions the meaningful control is a pre-execution gate on a defined class of high-impact operations, which general human monitoring of a running system cannot provide.

Article 14 does not require that gate.

Where the gap closes, or does not

Academic and policy work through 2026 has made this argument in more detail. Gardhouse, Oueslati and Kolt argue in Regulating AI Agents, posted in March 2026 and revised in July, that the Act was promulgated before the widespread use of agents, and that its allocation of monitoring and enforcement responsibilities, its reliance on industry self-regulation and its level of government resourcing suit conventional AI systems and not agentic ones. Nannini and colleagues, writing in April 2026 on the compliance architecture agent providers face, conclude that high-risk agentic systems with untraceable behavioural drift cannot currently satisfy the essential requirements of the Act.

Three routes could narrow the gap. The Commission could issue guidance, which is the fastest and which its Service Desk answer signals it is considering as developments warrant. A harmonised standard could address agentic architectures, though JTC 21 has published nothing on agentic security and no deliverable of any kind has been cited in the Official Journal. Or the Act could be amended, which the Digital Omnibus showed is possible: proposed on 19 November 2025, in force on 27 July 2026, roughly eight months for a change that deferred dates without rewriting requirements.

As of September 2026 none of the three has produced an agent-specific security control model, and the Annex III obligations apply from 2 December 2027.

What to do about it

Treat the Act as a floor and threat model separately. You need a separate assessment for the deployment, covering injection through retrieved content, memory integrity, tool permission scope and the blast radius of a compromised planning step. Article 15’s list names none of the four.

Scope credentials to the step. The highest-value control for agentic systems is that a compromised planning step cannot reach anything the specific task did not require.

Gate irreversible actions before execution. Decide which operations cannot be undone, and require an approval or a policy check on that class. A person watching a log is not a substitute.

Log at the action level. Article 12 requires automatic event logging over the lifetime of a high-risk system, and for an agent the useful unit is the tool call with its arguments and its result. That log is also the evidence base for any Article 73 serious incident report.

Document your role allocation and your reasoning. Where your agent calls third-party services, write down who you consider the provider and the deployer for each component, which Article 25 trigger applies, and where the answer is genuinely open. When guidance arrives, that document is what you update. When an incident arrives first, that document is what separates a defensible position from an improvised one.

The other four provisions of the AI Act that touch a security function are covered in the EU AI Act for security teams.

222fb9d292e3d0111656a33900e24a27cfb6a36eb7b202a94a66bb84766154b4?s=120&d=mp&r=g
[email protected] | About me |  Other articles

In the early 2000s, running emerging-technology risk labs at CyberAgency, a defence client asked my team to break the AI systems they planned to put into weapons. We did. That is where my work on AI security started, two decades before the current wave of attention. I kept at it through risk labs at IBM, Accenture, PwC and KPMG. In 2016 I co-wrote a book on AI and leadership. My commercial work today is quantum, at Applied Quantum, which is why this site sells nothing.