Mistral AI releases Leanstral 1.5: a free model for formal verification with record results
Mistral
OpenAI
Mistral AI has released Leanstral 1.5, a free Apache-2.0 licensed model with 119B total and 6B active parameters, achieving state-of-the-art results in formal verification. It saturates the miniF2F benchmark, solves 587/672 PutnamBench problems, and sets new records on FATE-H (87%) and FATE-X (34%). The model also uncovers real-world bugs in open-source repositories.
Mistral AI released Leanstral 1.5, a free Apache-2.0 licensed model with 119B total and only 6B active parameters, delivering a significant performance upgrade in formal verification. The model saturates miniF2F, solves 587 out of 672 PutnamBench problems, and achieves state-of-the-art results on FATE-H (87%) and FATE-X (34%). Training involved mid-training, supervised fine-tuning, and reinforcement learning with CISPO in two RL environments: a multiturn environment for theorem proving with Lean compiler feedback, and a code agent environment that operates like a developer in a raw filesystem. Leanstral 1.5 shows strong test-time scaling, with performance on PutnamBench improving from 44 problems solved at 50k tokens to 587 at 4M tokens. It also excels in code verification, proving time complexity guarantees for AVL trees and uncovering 5 previously unknown bugs across 57 tested repositories. The model is fully open-sourced via Hugging Face and a free API endpoint.
- Сокращения
- API = Application Programming Interface — интерфейс программирования приложений
- RL = Reinforcement Learning — обучение с подкреплением
Source: Mistral AI —
original