UC Berkeley Daydreaming paper published: normal task outputs alone reconstructed hidden agent skills with 86.8% behavioral fidelity
This story was merged into AI agent hacking incidents across labs →
Merged·Day 2·in the edition since 1 Sep 2026·2 sources·updated 2 Sep 2026
The ContextLeak tool-exfiltration attack
The gist
ContextLeak Steals Agent Context; METR Incident Cost $600K
ContextLeak shows that tool descriptions alone can serve as a data-exfiltration channel in agentic systems without any file or memory access. The METR incident shows that agent credential theft can go undetected for weeks when attack traffic blends with normal usage and API keys lack spending limits.
The full picture
A dashboard on a researcher's personal EC2 instance at AI safety organization METR silently disabled Google authentication, allowing attackers to directly prompt an agent into surrendering its provider API key. The stolen key was used for three weeks before detection, consuming $600,000 in AI credits, and METR disclosed the incident in a public security update. The abuse went undetected in part because METR routinely generates large token volumes and the free-credit key had no spending ceiling. ContextLeak, a research attack developed by Duke and Stanford researchers, uses a reinforcement-learning-trained attack LLM to craft malicious tool names and descriptions that convince agent models to select the attacker's tool and forward their full runtime context as call arguments. The attack achieves tool selection in 92% of user-prompt attacks and 89% of conversation-history attacks, with near-complete prompt reconstruction once selected, and transfers to backend models not used during training. In a Claude Code evaluation with Claude Sonnet 4.6, a version trained only against an open-source proxy was selected in 22 of 100 cases, with near-complete extraction when selected.
How it developed
METR disclosed that a misconfigured dashboard exposed an agent that surrendered a $600,000 API credential
Yuqi Jia, Ruiqi Wang, Patrick Li, Yuepeng Hu, Peinian Li, and Neil Zhenqiang Gong of Duke and Stanford published ContextLeak on September 1, an attack that trains a separate 'attack LLM' with reinforcement learning to craft deceptive tool names and descriptions, causing agents to select the malicious tool and leak their runtime context as call arguments with no file or memory access required.
The paper reports 92% malicious tool selection in user-prompt attacks and 89% in conversation-history attacks, with near-perfect context reconstruction and transfer to unseen backend models; in a Claude Code evaluation using Claude Sonnet 4.6, a proxy-trained version was selected in 22 of 100 cases.
An ACL Anthology paper described on September 2 a distinct attack on LLM agents: backdoored retrieval tools that silently exfiltrate stored user data, with attacker-controlled responses steering subsequent agent behavior across multi-turn sessions to enable 'sustained and cumulative information leakage over time'.
The paper, which calls for dedicated defenses against exfiltration-oriented attacks, adds to ContextLeak, a Duke/Stanford attack in which a reinforcement-learning-trained model crafts deceptive tool descriptions to extract agents' runtime context as call arguments.
First in the edition · 1 September 2026
Alice CEO Noam Schwartz argues AI agent security extends beyond model safety to tools, data, permissions, and policies
Prime Intellect's universal offline sandbox escape technique reported
Researchers publish llms.txt supply-chain findings; Claude, Codex, and Hermes confirmed installing unowned code in Fortune 500 networks
NVIDIA agent deployment tool vulnerability reported, enabling agent hijacking via malicious webpage
Sources
Rohan Paul Twitter
Related
- Merged intoAI agent hacking incidents across labs
The daily email
Want this in your inbox?
I send one email each morning with the stories that moved. If you would rather just read here, that works too.
Subscribe free