The First Known Out-of-Control AI Agent — or a Very Bad Marketing Stunt?

Simon Willison shares his thoughts on Martin Alderson's comment about an OpenAI AI agent's accidental cyberattack on Hugging Face. First, Hugging Face represents an extremely attractive target for finding vulnerabilities related to arbitrary code execution (ACE), due to the enormous number of interfaces running unverified models and code. Second, Willison is puzzled as to how OpenAI failed to notice that their sandbox had been so deeply compromised by the agent, and suggests that they may have been running numerous benchmarks simultaneously with essentially unlimited token budgets, testing different model checkpoints to assess its improvement during training.