66 Studies Reveal Big Blind Spots In How AI Agents Are Tested For Attacks
A review of 66 studies on AI agent security found that most testing focuses on a few kinds of attacks against single bots, leaving threats like attacks spreading between bots and slow, long-running compromises far less tested.
Key Takeaways
- Researchers at Australia’s CSIRO and South Korea’s Sungkyunkwan University reviewed 66 agent security studies published from 2022 to 2026
- They looked closely at 22 red-teaming studies that released their tools or data, and 11 security benchmarks
- Testing concentrates on prompt and reasoning attacks, memory attacks and attacks through tools, mostly against single agents
- Attacks across several cooperating agents, attacks through human users, persistent and long-running attacks got much less testing
The paper, posted to arXiv on Sept. 20, is by Heewon Baek and Hyoungshick Kim of Sungkyunkwan University and Alsharif Abuadbba, Kristen Moore and Surya Nepal of CSIRO, Australia’s national science agency.
Why Agents Need Different Security
An AI agent does more than answer questions. It makes plans, keeps memory between steps, uses outside tools and can work with people and other agents. That gives attackers many more ways in than a plain chatbot offers.
The authors give one example of how an attack can travel through an agent. Harmful text returned by an outside tool can mislead the agent’s reasoning, get saved into its memory and later push it into an action nobody approved. Looking at only one part of the system, they argue, hides that path.
What Gets Tested And What Gets Missed
| Area | How Much Testing It Gets |
|---|---|
| Prompt and reasoning attacks | Most tested |
| Memory attacks | Most tested |
| Attacks through tools | Most tested |
| Single-agent setups | Most tested |
| Persistent attacks | Less tested |
| Attacks between humans and agents | Less tested |
| Complex multi-agent setups | Less tested |
| System-wide and long-running attacks | Less tested |
The authors stress that these gaps describe the studies they selected, not every piece of research ever published. Still, the pattern matters because many companies are now connecting several agents together and letting them run for hours, exactly the setups that get the least testing.
Also Read: Apiiro Finds 2,539 Secrets Leaked By Claude Code Users On GitHub
Hard To Compare, Hard To Repeat
The review also found that the evidence itself is hard to weigh. Studies use different ways of measuring success, rarely test defenses that adapt to the attacker, lean toward some agent designs over others and often fail to record exactly what the agent did at each step.
That makes it difficult to compare one result with another or to repeat an experiment, which is how security claims normally get confirmed. To track this, the authors logged which studies released their code, attack prompts, datasets and test scripts.
What The Researchers Want Next
The paper ends with 13 open research questions aimed at making agent security testing more systematic and easier to reproduce, including more work on multi-agent systems, threats that pass through human users and attacks that unfold over long tasks.
For companies rolling out AI agents, the takeaway is simple. A product passing today’s common security tests says little about how it holds up when several agents work together or run unattended for a long time.
Read Next: Anthropic’s Claude Code Has Been Sending Users’ Email Addresses To Websites
