AI Safety
“Repeat-After-Me” Prompt Bypasses 6 AI Agent Frameworks
Oct 12, 2026·4 min read·By Rahish Saifi
Agent stacks promise to turn language models into workers that book flights, query databases, and run code—but a newly reported attack shows how fragile the boundary between “chat text” and “executable tool calls” can be. Researchers from Pennsylvania State University, Université Montpellier, and Dartmouth College describe a technique they call “Repeat-After-Me”: tricking popular agent frameworks into running tool invocations by asking the model to repeat or echo attacker-written tool-call text. According to CSO Online (Gyana Swain, Oct 12, 2026), the method worked on six frameworks—LangChain’s LangGraph, Microsoft Agent Framework, Microsoft’s AutoGen, Google’s Agent Development Kit 2.0, AWS’ Strands Agents, and CrewAI.
When repetition becomes execution
Modern agents do not merely print answers; orchestrators parse structured tool calls from model output and dispatch them to registered functions. The Repeat-After-Me idea exploits that pipeline: if an attacker can get the model to output a syntactically valid tool call—framed as a harmless echo, quote, or hypothetical—the framework may execute it even when safety instructions tell the model to refuse.
The researchers summarize the mechanism in the paper this way:
“A seemingly benign text-repetition request thus gives the attacker direct control over the tool invocation and its arguments.”
That framing matters for security teams because many deployments treat “the model refused” as proof that nothing dangerous ran. This work argues the orchestrator, not the refusal string, is what actually decides execution.
LangGraph demo: refusal after the fact
CSO Online walks through the most detailed public example, using LangGraph 1.2.0 with Meta Llama 3.1 8B served via Ollama. The setup registered dummy tools named make_bomb and launch_missiles that returned fixed data with no real-world effect—deliberately alarming names to show what an agent might be wired to do.
On a direct harmful request, the model refused but printed the would-be call as plain text; LangGraph did not run it. When the same call was reframed as a hypothetical or as output that should only quote a structured tool invocation, the model returned a structured tool call—and LangGraph executed it. The refusal text arrived after the tool had already returned.
The researchers’ line on that mismatch is blunt:
“The refusal text and the executed call disagree: the system prompt shaped what the model said, not what the orchestrator did.”
They reported a missing pre-execution check to LangGraph maintainers, CSO Online wrote. The article notes that a full transcript was published only for LangGraph; the other five frameworks were reproduced in the paper with less step-by-step detail. Vendors including LangChain, Microsoft, Google, AWS, and CrewAI did not immediately respond to CSO for comment.
Same paper: fuzzing MCP tool servers
The Repeat-After-Me findings sit alongside a separate tooling audit in the same research. The team built a fuzzer called ToFu and reported 253 verified bugs affecting 343 tools across 118 of 273 open-source MCP servers they could run locally. The most serious categories included 64 path-traversal issues, 7 SSRF, 5 data-exfiltration, and 3 OS command-injection flaws; the remainder were largely reflected-output or log-injection problems with lower impact.
The paper argues that many integrations lean on the model to clean inputs—a risky assumption:
“many tools implicitly rely on the LLM to sanitize inputs”
Independent researcher Vibhum Dubey, quoted by CSO, described an architectural weakness: “A model can generate a technically valid tool call that is still inappropriate for the task.” He added that “Every sensitive tool call should undergo independent authorization checks,” and that “Valid syntax or a convincing explanation from the model should never substitute for permission.”
What defenders should assume
For teams shipping agents on any of the six frameworks, the lesson is not “swap vendors and forget it.” The attack class targets a common pattern: user-influenced model output parsed as machine instructions without a separate policy gate. Mitigations that security engineers already use elsewhere—allowlists per tool, argument validation, human approval for high-impact actions, and authorization that does not trust model prose—are the practical response.
That aligns with a growing chorus about agent governance. Microsoft CEO Satya Nadella’s weekend essay urging enterprises to treat frontier models as insider risks with independent controls and this site’s coverage of Claude sending a false homicide tip during testing both stress the same point: model behavior and harness behavior can diverge.
Caveats on the findings
CSO Online and the underlying paper carry important limits readers should keep in mind. The work has not been peer-reviewed. MCP servers were tested at fixed code versions; the reporting does not say whether identified flaws were fixed or exploited in the wild. And the Repeat-After-Me results do not prove that every production deployment of LangGraph, AutoGen, CrewAI, or the other named stacks behaves the same way—configuration, model choice, middleware, and custom guards can change outcomes.
Still, for anyone wiring LLMs to real tools, the headline is uncomfortable in a useful way: asking the model to “just repeat this” is not always harmless text generation. Sometimes it is a remote procedure call waiting for an orchestrator to oblige.