Google DeepMind Introduces SL2T, a Sign Language Translation Model Powering Sign-to-Text on Pixel 11
Google/DeepMind
DeepMind
Google DeepMind has unveiled SL2T, a sign-language-to-text translation model, and integrated it into Gboard and Live Transcribe on Pixel 11 for American Sign Language to English. The model is trained on over 100,000 hours of data across more than 50 sign languages and achieves a zero-shot BLEURT score of 70 on FLEURS-ASL. Developed with the Deaf community, SL2T translates from pose landmarks to text, supporting real-world use.
Google DeepMind introduced SL2T, a massively multilingual sign-language-to-text translation model, powering new sign language features for Deaf and hard of hearing users. It is the first consumer product integration, with sign-to-text dictation in Gboard and Live Transcribe on Pixel 11, starting with American Sign Language (ASL) to English. The model is trained on over 100,000 hours of data across more than 50 sign languages, with about a quarter in ASL. SL2T translates directly from pose landmark sequences to text, bypassing glosses, and uses on-device tracking (MediaPipe Holistic) for privacy. According to benchmarks like FLEURS-ASL, SL2T achieves a zero-shot score of 70 BLEURT. The development was community-driven, with the AI Sign Language Advisory Committee (AISLAC) guiding responsible deployment. Future plans include expanding to additional sign languages and sign language generation.
- Abbreviations
- SL2T = sign-language-to-text — перевод с жестового языка в текст
- ASL = American Sign Language — американский жестовый язык
- BLEURT = Bilingual Evaluation Understudy with Representations from Transformers — метрика оценки качества машинного перевода
- FLEURS-ASL = Few-shot Learning Evaluation of Universal Representations of Speech for American Sign Language — бенчмарк для оценки перевода с ASL на английский
- AISLAC = AI Sign Language Advisory Committee — консультативный комитет по ИИ для жестовых языков
Source: Google DeepMind —
original