QuantCode-Bench: a new benchmark for evaluating LLMs' ability to generate executable algorithmic trading strategies

QuantCode-Bench is a benchmark designed to evaluate LLMs' capability to generate executable algorithmic trading strategies from natural language descriptions. It contains 400 tasks sourced from Reddit, TradingView, StackExchange, GitHub, and synthetic data, covering three difficulty levels. The evaluation pipeline consists of four stages: Compilation (syntactic check), Backtest (runtime execution), Trade (at least one trade executed), and Judge (semantic alignment via an LLM judge). Two modes are supported: single-turn (one-shot generation) and agentic multi-turn (up to 10 iterative refinements with feedback). Results show that while nearly all strong models achieve perfect compilation in single-turn mode, the final Judge Pass rates vary widely, with GPT-5.5 leading at 82.5%. In multi-turn mode, performance improves dramatically, with GPT-5.5 reaching 98.8% Judge Pass. The benchmark highlights that syntactic correctness is insufficient for domain-specific code generation, and that models must understand the trading domain, API semantics, and operationalize vague descriptions.