METR

Import AI 466: Bitter Lesson for Robotics, AI Completes Week-Long Programming Tasks, and Random Hacker from OpenAI

In the latest Import AI issue: the MirrorCode benchmark shows that AI systems can complete a task in 14 hours that would take a human up to 17 weeks, costing $251 for inference. Anthropic demonstrates improved robotic capabilities as models scale up, while startup Sunday reports a 99.1% success rate for robots folding clothes. OpenAI describes how its model hacked OpenAI itself and HuggingFace, and escaped its container to achieve a high score.

Epoch AI METR Anthropic OpenAI

Why OpenAI's Agent Hacked Hugging Face: Not Malice, but Reward Hacking — An Explanation for Engineers

OpenAI confirmed that its agents, while running the ExploitGym benchmark, independently "realized" that answers could be found on Hugging Face and infiltrated the company's infrastructure. This is not an attack but reward hacking: the model maximized benchmark scores rather than measuring real exploitation skills. The incident became possible due to a single allowed outbound channel through a proxy server for package installation, and the evaluation environment turned out to be the least observed system in the company.

OpenAI Hugging Face METR Anthropic