OpenAI Field Report: AI agents rewrite fragile research software, but researchers must rigorously verify

A field report by OpenAI and academic partners documents eight case studies, mostly in biology, where coding agents such as Codex and Claude Code were used to maintain, optimize, or rewrite research software. Projects ranged from simple maintenance to complete rewrites in modern languages. For example, GPT-5.5 modernized the build process for the Python library cyvcf2, while two agents, Claude Code and Codex, migrated about 10,000 lines of MHCflurry from TensorFlow to PyTorch. The rustar-aligner project rewrote STAR, a multi-thousand-line C/C++ tool, in Rust, achieving 99.815% and 99.883% agreement with the original on test datasets. RustQC, which bundled 15 quality-control tools, reduced runtime from 15 hours 34 minutes to 14 minutes 54 seconds—a factor of over 60. HelixForge, a replacement for BamSurgeon, was 59.6 times faster overall, with the core computation 98.6 times faster. However, agents often appeared confident but were wrong; for example, in the bayesm rewrite, two procedures contained subtle errors, including an inverted control parameter and a faulty scaling factor. Researchers concluded that humans must define goals, success criteria, and validation methods, while agents handle implementation. The report estimates significant time savings—for instance, maintaining NumPy could save about 650 hours per year—but highlights long-term maintenance and responsibility as key challenges. Some changes were merged upstream, while others, like FastQC, were rejected by the original author, leading to improvements being incorporated into the original Java version instead. The report is retrospective and based on self-reports, and it notes that the bottleneck is shifting from implementation to validation and scientific review. This pattern is echoed by a METR study and a study on developer teams' frustration with AI code. The report aligns with OpenAI's broader science strategy, which includes a dedicated team led by Kevin Weil and the release of GPT-Rosalind.