Science One Framework: A verifiable autonomous research framework via Chain-of-Evidence
Google/DeepMind
Sakana AI
Google Research introduces the Science One Framework, an autonomous research prototype that eliminates hallucinations by building verifiable evidence chains, and the CoE Audit, an automated protocol to evaluate integrity of AI-generated papers. Results show Science One achieves zero phantom references and fully verifiable scores while matching or exceeding human expert performance on benchmarks.
Google Research has introduced the Science One Framework, an autonomous research prototype designed to eliminate hallucinations by natively building verifiable evidence chains, along with the CoE Audit, an automated protocol to evaluate the integrity of AI-generated papers. The framework is based on Chain-of-Evidence (CoE), a new verifiability concept that requires every claim in a research artifact to have a recorded evidence chain (completeness) and each chain to genuinely support the claim (correctness). The Science One Framework instantiates CoE through three main modules: an Idea Explorer, a Problem Investigator, and a Solution Developer. The CoE Audit applies four strict integrity checks: reference verification, score verification, code reproducibility, and method-code alignment. In evaluations on 75 papers from the ADRS benchmark, Science One achieved zero phantom references compared to baseline hallucination rates up to 21%, perfect score verification, and the highest method-code alignment, while matching or exceeding human expert performance on all five ADRS tasks. The framework was also tested on six external complex tasks including MLE-Bench and Parameter-Golf, achieving state-of-the-art results.
- Сокращения
- CoE = Chain-of-Evidence — Цепочка доказательств
- ACID = Atomicity, Consistency, Isolation, Durability — Атомарность, согласованность, изоляция, долговечность
- ADRS = Automated Design of Research Systems — Автоматизированное проектирование исследовательских систем
Source: Google Research —
original
