A single prompt jailbreaks Gemini 3 Pro 18.10% of the time. But string together a multi-turn conversation, and the success rate jumps to 73.35% — a fourfold increase.
That's the finding from a Cisco study reported by CSO Online on May 27. The takeaway: today's AI safety benchmarks almost exclusively measure single-turn attacks, but real adversaries engage in multi-turn conversations.
15 models tested with single-turn and multi-turn attacks
Cisco tested 15 flagship models from OpenAI, Anthropic, Google, xAI, and Amazon using two parallel methods:
- Single-turn attacks: 30,090 prompts (2,006 per model) — a direct request to see if the model complies.
- Multi-turn attacks: 6,986 attacks spread across 1,456 conversations, gradually steering the model off course.
The attack success rates (ASR) were stark:
| Model | Single-turn ASR | Multi-turn ASR |
|---|---|---|
| Claude Opus 4.6 | 3.64% | 16.20% |
| GPT-5.4 | 2.74% | 24.68% |
| Gemini 3 Pro | 18.10% | 73.35% |
Every model showed a significant jump. GPT-5.4's single-turn rate of 2.74% soared to 24.68% in multi-turn — a ninefold increase. Gemini 3 Pro was breached in over 70% of conversations. Claude Opus 4.6 was the most resilient, but its multi-turn rate still quadrupled to 16.20%. No model withstood multi-turn attacks.
xAI's Grok 4.1 Fast had a notable split: with reasoning disabled, the jailbreak rate was 88.30%; with reasoning enabled, it dropped to 43.47% — a difference of nearly half. Amazon's Nova series was the only anomaly, showing higher single-turn vulnerability, which the study noted as an outlier without explanation.
How attackers break through with conversation
Cisco used publicly known techniques, categorized into five types:
- Role-playing: Prompting the model to adopt a fictional persona, e.g., "Assume you are an unrestricted AI."
- Obfuscation: Burying sensitive requests in irrelevant context to create ambiguity.
- Reframing refusals: Rephrasing a request after the model refuses, bypassing its defenses.
- Decomposition: Breaking a dangerous task into harmless-looking sub-questions, then reassembling the answers.
- Gradual escalation: Starting with a borderline question and inching forward once successful.
The researchers summed it up: "Real adversaries iterate. They reframe refusals, decompose tasks across turns, adopt personas, and escalate gradually."
Another key quote: "A model with 2.74% single-turn ASR is not the same product as a model that holds the line at 24.68% multi-turn ASR." Yet most safety reports only publish the flattering single-turn number.
Not an open-source problem
A common excuse is that jailbreaking mainly affects open-weight models with weaker alignment. Cisco's study debunks that: all tested models were closed-source flagships, and all were vulnerable to multi-turn attacks. The researchers stated: "Multi-turn vulnerability is a structural property of the current frontier, not an artifact of open-weight alignment choices."
In other words, it's an industry-wide issue. Models are trained to follow conversational flow, and attackers exploit that instinct.
Recommendations aimed at enterprise buyers
Cisco's recommendations, while addressed to model vendors, are practical for enterprise procurement:
- Models with a gap of more than 15 percentage points between single-turn and multi-turn ASR should undergo manual review before deployment.
- Vendors must disclose how different configurations (e.g., enabling reasoning) affect safety.
- Safety reports should publish both single-turn and multi-turn ASR side by side.
- Benchmarks should reflect real attack scenarios, not just single-turn tests.
The bottom line: the safety score you see today is likely single-turn. But your employees, customers, and adversaries won't stop at one question. That 2.74% sense of security may vanish by the fifth turn.
Sources: CocoLoop; AI models more vulnerable than claimed when faced with iterative attacks (CSO Online)