Sber Opens Access to SIRIN Library for Detecting Errors in AI Responses

The team of scientists from Sberbank's Center for Practical Artificial Intelligence has published the SIRIN (Semantic Inconsistency Recognition and Inspection Nexus) library as open source. SIRIN helps make AI agents and language models more reliable by identifying fabricated facts or those not supported by source data. The library merges various response verification methods, allowing developers to compare, combine, and choose the most suitable approach for their tasks. It can check entire responses or individual text fragments and indicate where the model may have erred. Additionally, SIRIN can preliminarily assess whether the AI agent has sufficient information to answer, helping to prevent errors by requesting more data or declining to answer. The library is based on Sber's research in hallucination detection, particularly metamodels that improve error detection accuracy by nearly 30% using only 250 training examples. Sergey Ryabov, Senior Managing Director and Director for AI Transformation at Sberbank, noted that the approach has proven effective internally and is now open to all companies to foster market development. Developers can integrate SIRIN into their assistants within minutes, and it is useful for businesses implementing large language models, as well as developers of AI assistants, RAG systems, and AI agents. The library is available on GitVerse, GitHub, and a web demo on Hugging Face.