Google Research: New Methodology for Assessing 'Future Skills' Using Generative AI

Google Research has introduced Vantage, a research experiment that uses generative AI to assess 'skills of the future.' Developed in partnership with educators and researchers from New York University, Vantage is designed for high school and college students. The platform simulates conversations with AI avatars in a controlled environment. Users encounter open-ended scenarios, such as preparing for a debate or presenting a creative idea. An Executive Large Language Model dynamically analyzes the conversation flow and introduces specific challenges—objections, conflicts—to gather sufficient information for evaluation. After the task is completed, an AI analyzer reviews the transcript and scores it according to pedagogical rubrics, providing users with a visual skills scale and qualitative feedback. In a joint study with 188 volunteers (ages 18-25, in the US), the scores from the AI analyzer were compared with those from NYU experts. The agreement between AI and experts was found to be comparable to that between two human experts. A study was also conducted with OpenMic: the AI analyzer evaluated creative work from 180 students (character interviews, articles)—the correlation with expert assessments was high. Vantage is now available for registration on Google Labs and currently operates in English. The developers plan further research on the transferability of skills from simulation to real-world interactions, as well as evaluating the effectiveness of practice in simulated environments.