Age of Autoresearch: I Went to Sleep and 40 Hypotheses Were Tested Overnight — What Should I Do and How to Work Next?

An engineer describes how, instead of manually testing hypotheses, he wrote a text description of the hypothesis generation pipeline, asked Gemini to judge each attempt, and let an agent run overnight. In the morning, the log showed 40 hypotheses tested. He contrasts his early naive attempts (running ten agents in parallel without a plan) and his second attempt (giving agents a detailed agenda) with the lazy evening approach that worked. He explains that the key was not better control of agents but who evaluates the result. He cites precedents: sankalp's QR decomposition achieves a 232x speedup on batched QR, Karpathy's 630-line autoresearch script ran hundreds of experiments overnight, and DeepMind's AlphaEvolve improved solutions for 20% of 50 open math problems and accelerated Google's datacenters, chip design, and AI training. He shares two personal cases: in the first, he spent two weeks hand-crafting dataset filters, but a Gemini-based oracle with a VLM beat that in a couple of runs; in the second, he got a 30% training speedup but hit pitfalls: he forgot to include a loss parity check (leading to NaN loss), and an insufficiently detailed oracle missed artifacts like leftover letters in image-text removal. He emphasizes that building a good oracle and loop is the real work, and that the model inside the loop matters. He concludes that expertise is shifting from "find a solution" to "build a system that finds solutions".