Gray Swan Targets $300K Agent Jailbreaks After OpenAI And Anthropic Stress Tests

Gray Swan relaunched its Arena contest to target AI agents, offering $300,000 in prizes for techniques that can jailbreak tool-using systems from leading model developers.

Key Takeaways

• Gray Swan relaunched its Arena contest to target AI agents and offer $300,000 in prizes for jailbreak techniques
• Fortune reported the company rebuilt the contest after helping Anthropic, OpenAI and Google DeepMind stress-test models before release
• Arena shifts focus from chatbot prompt jailbreaks to agent-specific attacks aimed at the chain from reasoning to action
• The contest will test the whole stack including agent wrappers, browser connectors and API rules, not only model text output

Fortune reported that the company rebuilt the contest for the agent era after helping companies including Anthropic, OpenAI and Google DeepMind stress-test models before release, shifting the focus from chatbot prompts to systems that can take actions.

That change matters because AI agents expand the attack surface, as a chatbot that produces bad text can mislead a user, while an agent that can browse, write code, call tools or access accounts can do more damage if an attacker pushes it past its rules, moving the security problem from harmful words to harmful actions.

Gray Swan Relaunch

Gray Swan operates in the red-team market for frontier AI, where red teaming means organized attempts to break a model’s safeguards before customers do. With plain chatbots, that often meant prompt tricks that generated banned text.

With AI agents, the test gets harder because an agent can inherit the model’s weaknesses and add new ones from its tools.

If the model misreads a web page, follows a malicious instruction in hidden text or calls the wrong API, the failure can spill into live systems. That is why Arena is shifting from generic jailbreaks to agent-specific attacks aimed at the chain from reasoning to action.

The $300,000 pool is also a buying signal for customers deploying agents into finance, health care or internal systems, but the amount alone is not a measure of coverage or rigor.

Evaluating what that prize pool buys requires examining the contest rules, threat models and scoring criteria, which determine whether reported successes map to realistic attack paths or narrow, contrived scenarios.

The public value of the contest depends on whether it yields reproducible failure modes that labs and vendors can use to harden deployments.

Also Read: Ready Server Targets VPS Teams Facing Token-Bill Risk With Private LLM API

From Prompt Hacks To Tool Abuse

The relaunch targets a documented shift in what security testing must address, moving from harmful outputs to action-oriented failures tied to tool use and permissions. The first wave of public testing centered on harmful outputs, bias and policy evasion.

The second wave is about action security, where the main question is not what the model says but what it is allowed to do.

That distinction becomes more urgent as labs and software vendors attach tools to models, since an AI agent that can search internal documents, draft purchase orders or trigger workflows increases the surface for prompt injection, tool misuse or hidden instruction attacks.

The contest format gives independent researchers an incentive to publish working attack methods so labs can test whether their controls stop those methods before commercial rollouts widen.

How Agent Jailbreaks Differ From Chatbot Hacks

Agent jailbreaks target the bridge between reasoning and action. A chatbot jailbreak usually tries to make a model say something disallowed.

An AI agent jailbreak tries to make a system do something it should not do, like follow a poisoned webpage, exfiltrate data or misuse a connected tool.

What The Contest Will Actually Test

That distinction helps explain why companies like Anthropic and OpenAI have shifted attention toward permissions, tool scopes and monitoring, since the model is only one layer, and the agent wrapper, browser connector and API rules matter just as much.

Arena is positioned as a public stress test for that whole stack, not only for the model’s text output. The next thing to watch is whether the organizer publishes attack categories and participant names, since the usefulness of the exercise will hinge on whether it produces reusable failure patterns rather than one-off stunts.

Read Next: AI Is Coming For Your Checkout As Store Card Giant Synchrony Signs Up OpenAI