Google Research: New Method to Assess 'Future-Ready' Skills with Generative AI
Google/DeepMind
Google Research, in partnership with New York University, has developed Vantage, a research experiment that uses generative AI to assess 'future-ready' skills like critical thinking and collaboration. The AI scoring system matches human expert ratings, and the platform is now available on Google Labs for English-speaking high school and college students.
Google Research, in collaboration with New York University, has introduced Vantage, a research experiment that leverages generative AI to assess 'future-ready' skills such as critical thinking, collaboration, and creative thinking. Vantage places learners in dynamic, multi-party conversations with AI avatars to complete tasks, while an Executive LLM steers the dialogue to gather assessment information. An AI Evaluator then analyzes the conversation against a rubric and provides a detailed skill map with scores and qualitative feedback. In a study with 188 testers aged 18-25, the AI Evaluator's scores showed agreement with human experts comparable to inter-rater reliability. A separate study with startup OpenMic on creative tasks also showed high correlation between AI and human scoring. Vantage is now available for sign-up on Google Labs in English.
- Сокращения
- LLM = Large Language Model — Большая языковая модель
- GenAI = Generative Artificial Intelligence — Генеративный искусственный интеллект
- OECD = Organisation for Economic Co-operation and Development — Организация экономического сотрудничества и развития
- WEF = World Economic Forum — Всемирный экономический форум
Source: Google Research —
original