Well, it happened again. There I was scrolling through LinkedIn and another piece of AI news caught my eye which I knew I was going to cover in this newsletter, as it was so up my street. For those that are not familiar, OWASP is a community driven body which has long held the top spot for application security guidance. In the rise of AI they first released the OWASP Top 10 for LLMs, the 10 most common and harmful security mistakes made in LLM deployments. Whilst this interested me at the time, it was only on the surface level due to it featuring heavily on LLMs themselves whereas my growing interest was in agentic. Well, the time has come that they have now released an agentic application security guide, complete with a plethora of tangible security controls across all layers of agentic AI systems.

With my secure agentic AI adoption framework (DAMSIC) being released only last week this felt like perfect timing to cross reference with how OWASP were looking at a similar problem. So, to save you reading the full 81-pages on OWASPs thoughts I'll break down the core security domains, noted threats, and tangible security recommendations. That said, this really is a deep dive into the weeds of secure AI system deployment, so trying to cover everything (even at the surface level) would take us far beyond the scope of a newsletter.

The document is broadly broken down into:

1. Introduction

2. Secure Agentic Architectures

3. Agentic Developer Guideline

4. Enhanced security actions for Agentic AI Systems

5. Key operational capabilities

6. Agentic AI and Supply Chain

7. Assuring Agentic Applications

8. Secure Agentic Deployments

9. Runtime hardening

For those who follow this topic closely you'll recognise a good number of them, but perhaps here with more meat on the bones with regards to tangible best practices. Each of these domains is mapped against a list of 15 prominent threats, referred to as T1-15. These include: memory poisoning, privilege compromise, tool misuse, goal manipulation, cascading hallucination, rogue agents, human attacks, communication poisoning, overwhelming HITL, identity spoofing, to name a few. Quick note here, I found it interesting that they raised an overwhelming amount of human oversight / intervention (HITL) as one of the risks - definitely a lesser thought of one!

So, let's start going through these high-level domains, capturing the main talking points and providing some clear examples of controls that should be considered for each area.

Secure Agentic Architectures

This was a brilliant section in my eyes, and really lays out how we should be thinking about AI systems from a security perspective. It goes without saying these days that mapping out how everything will work and threat modelling it right from the start will avoid big costly (or harmful) errors down the line. The guide starts by defining some key language that we'll be using here, before starting to map out the attack surface components for these systems, like the LLM, orchestration control flow, memory, etc. It then goes on to show secure examples of tool usage flows, as well as describing a few different multi-agent architectures and how they work, such as hierarchical or swarm. Finally, they call out the various frameworks used for agent orchestration and connecting the dots, and giving a case study for a multi-agent system architecture

Whilst this may not be strictly security related, laying out in their own words how architecture can affect security from the roots is a very mature way of looking at security and one that resonates with me. For example, they heed caution regarding customising frameworks (LangChain for example) too much, due to the fact that should that framework be patched for a vulnerability a client's implementation may no longer fit with the updated core and could cause significant impact. Whilst many may not think of this as a traditional 'cybersecurity' challenge, I urge you to recall the A in the CIA triad.

Agentic Developer Guidelines

This is where we start to build in secure-by-design principles which exist from initial design all the way through to ongoing security operations. We start with threat modelling some of the new and existing attack surfaces that developers should be aware of, before jumping into concrete secure development practices: input validation, error handling, input and output guardrails and human-in-the-loop.. Naturally many of the more traditional security recommendations exist here too, such as authorisation, SAST, vulnerability scanning, JIT, pentesting, continuous monitoring, etc. Finally this section touches on secure runtime operations, which aligns with my own thoughts regarding limiting autonomy, transparency in decision making and session scoping.

Enhanced Security Actions for Agentic AI Systems

This is where we start to get into the real nitty gritty recommendations (nay, requirements) that these systems will need, broken down into single agent and multi-agent. Naturally, best practice here calls out things like OAuth 2.0, managed identities, RBAC, least privilege, data encryption, data labelling, dependency monitoring, real-time alerting, kill switches, etc. This really did feel like the laundry list of tangible controls to keep in mind when building this stuff, which I like as a practitioner. Many of the pieces of research, legislation, guidance, etc. up to this point have remained super high-level. For example, I remember trying to go through the EU AI Act and draw up some tangible security controls and recommendations from it and it was like getting blood from a stone. Here on the other hand we get a low level walk through of various aspects of the security ecosystem: authentication, authorisation, data protection, code security, monitoring & IR, prompt security, inter-agent communication, trust boundaries and much more. Side note, I am happy to see that they specifically call out input guardrails AND output guardrails as two separate requirements, as my experience of testing production AI systems is that the latter is rarely thought of.

Key Operational Capabilities

This was a brilliant section too. Cognisant of the fact that with AI Agents we are routinely hooking them up to things like APIs, databases, IDEs and web browsers, this section was all about the gotcha's we should be looking for with these common tool calls. Going tool by tool they lay out what things to watch out for:

• API - unauthorised data access, exposed keys and DoS

• Code execution - RCE, code injection, information leakage

• Database interaction - SQLi, data exposure, RAG poisoning

• Web - malicious web content (XSS), phishing, SSRF, browser data exposure

• PC Operations - unauthorised file system access, OS command execution, lateral movement

• Critical systems - catastrophic physical outcomes, malicious control injection, influencing decisions

For each of these items we get a cherry picked list of 3-4 controls that we need in place to mitigate the biggest risks. API? Least privilege. Code exec? Sandbox. Database? safe queries, etc. etc. Yet more well thought through, tangible advice which will help designers, architects and testers stay ahead of proven risks with AI agents.

Supply Chain

For those that read my secure agentic AI adoption framework (DAMSIC) you'll be familiar with supply chain being a troublesome component (as ever btw) in AI systems too. This is a short but sweet section without any fluff, just some key things to be aware of in supply chain scenarios:

• Code security is key. Whether its a third-party library or LLM-generated we'll need to the more typical controls like SCA scanning, and the more agentic challenges like HITL for high-risk code execution

• Keep environments (dev vs prod) separate and used for their intended purposes.

• Maintain agent and tool registries, ensuring you know when / if things are changing

Assuring Agentic Application

This is where we enter into my world - security testing. It's nice to see my dear red teaming getting a feature here. Red teaming and general security assurance designed to proactively uncover things like prompt injections and privilege escalation need to be built into the release framework, and we get some more great tangible guidance on breaking down the current tools which can be used to help prevent many of these attacks.

I still cringe every time I see benchmarks being mentioned in the same breath as red teaming, but they do indeed call out mapping agent behaviour against benchmark datasets as fundamental elements for standardising performance evaluation. More in-line with traditional red teaming however is behavioural testing, which evaluates agent activities and interactions at runtime for potential security flaws, rather than just the underlying code.

Putting my own definitions of red teaming aside, I do think this is good to be called out, and I think with the academic history of AI we will always see elements like benchmarks. My view is that red teaming to secure agentic systems will need to find the right balance of quantitative and creative. Only then are you truly making meaningful security enhancements.

For those who are perhaps less benchmark-inclined (me), they have included a step-by-step guide which is great:

1. Define security objectives - lets availability

2. Assess the threat landscape - DoS

3. Research existing benchmarks - look for benchmarks addressing agentic AI vulns such as excessive resource consumption, in this case

4. Evaluate benchmark criteria - does it align with your objective? is it a recognised and accepted benchmark?

5. Test benchmark suitability - false positives, etc.

6. Compare and select benchmark - based on coverage, relevence, effectiveness, etc.

7. Establish continuous evaluation - retest with updated benchmarks, integrate benchmark testing in CI/CD

8. Engage with AI security community

Secure Agentic Deployments

Now we get to the thorny bits.deploying agents in production. They call out several of the main principles we should be aware of:

• Secure pipelines and rogue agent checks - code signing, provenance tracking for agent versions and manual reviews of agents handling high-risk tasks is a starter for 10

• Role containerisation - assuming the worst (a compromised agent) we want to limit the blast radius by isolating agent roles, segmenting networks, and using ephemeral execution contexts

• API security - access controls, rate limits and API gateways are just some of the things we'll need to look into when security external tools access via APIs. These are critical security boundaries

• Alerting - anomalous behaviour should be identified in realtime and alerted upon. This has been a core security tenet and its no different here

• Human oversight - where critical decisions and high-risk actions are being made, seek human review and / or approval before execution.

Runtime Hardening

Finally, we need to make sure the infrastructure that our agents are running on have been fully locked down. Hardening VMs, network isolation, sandboxed runtimes, memory and state hygiene, observability and forensics, session management and more feature here. Again, locking down the infrastructure is nothing new to us, but another avenue where things can go wrong with agents if we take our finger off the trigger.

Conclusion

Well, if you are like me you might have started to become just slightly overwhelmed with the amount of depth of the security approach that we are seeing in guides like this and other similar ones. Just doing one of these security domains exhaustively will be a considerable job, and there are 8 of them.all of which need to be in place before we can call something 'secure'. However, with every new guide, piece of legislation and code of conduct we are getting closer to having a shared understanding of what 'good' looks like. That was always going to be the first challenge, and one I'd say that we are making very good progress with. Now, the challenge will be widescale adoption of these security standards. Whilst that journey is going to be long and no doubt fraught with complexities, the path ahead is clear. Now we just need to go and do it.

Hope you enjoyed, and catch you next week!