Measuring the impact of learning with AI in Sierra Leone and beyond

Google DeepMind released results from an eight-week randomized controlled trial in Sierra Leone, evaluating Guided Learning in Gemini with 1,763 junior secondary students across 12 schools. The trial, supported by Fab AI and the Sierra Leone Ministry of Education, found that students using the tool saw a +0.258 standard deviation gain in math scores, equivalent to 1.2-1.7 years of typical learning. Students whose teachers integrated Gemini into about half their lessons achieved gains of 1.8-2.5 years. Engagement was high, with 69% meeting or exceeding usage targets, far above the typical 5% for educational technology. The tool was designed to foster understanding rather than providing direct answers; 91.4% of student interactions focused on building conceptual understanding, and Gemini provided direct solutions in only 2% of cases. Teachers reported a shift from lecturers to facilitators, and the tool supported their professional growth. The study also revealed an achievement gap, with stronger students benefiting more. DeepMind plans to expand trials to other countries and explore areas like metacognition. The research received support from Google.org and the Gates Foundation, with collaboration from EducAid, Laterite, and Oxford MeasurEd.